CachingCaching*
*



Contents  *



Index  *Topic Contents
*Previous Topic: Cookie Functions
*Next Topic: Appendix A: HINTERNET Handles

Caching

The Win32 Internet functions have simple, yet flexible, built-in caching support. Any data retrieved from the network is cached on the hard disk and retrieved for subsequent requests. The application using the Win32 Internet functions can control the caching on each request. For HTTP requests from the server, most headers received are also cached. When an HTTP request is satisfied from the cache, the cached headers are also returned to the caller. This makes data download from Win32 Internet functions seamless, whether it is coming from the cache or from the network.

Unlike previous versions of the Win32 Internet functions, the current and future versions do not have an upper limit on the cache entry size. Applications must properly allocate a buffer in order to get the desired results when using the persistent URL caching functions. For more information, see Appendix B: Using Buffers.

Using Flags to Control Caching

The Win32 Internet function flags allow an application to control when and how it uses the cache. These flags can be used alone or in combination with the dwFlags parameter in functions that access information or resources on the Internet. The Win32 Internet functions store all data downloaded from the Internet by default.

The following flags can be used with the Win32 Internet functions to control caching:
INTERNET_FLAG_DONT_CACHE Does not cache the data, either locally or in any gateways. Identical to the preferred value, INTERNET_FLAG_NO_CACHE_WRITE.
INTERNET_FLAG_HYPERLINKForces the application to reload a resource if no expire time and no last-modified time was returned when the resource was stored in the cache.
INTERNET_FLAG_MAKE_PERSISTENTNo longer supported.
INTERNET_FLAG_MUST_CACHE_REQUEST Causes a temporary file to be created if the file cannot be cached. Identical to the preferred value, INTERNET_FLAG_NEED_FILE.
INTERNET_FLAG_NEED_FILE Causes a temporary file to be created if the file cannot be cached.
INTERNET_FLAG_NO_CACHE_WRITERejects any attempt by the function to store data downloaded from the Internet in the cache. This flag is necessary if the application does not want any downloaded resources to be stored locally.
INTERNET_FLAG_OFFLINEPrevents the application from making requests to the network. All requests are resolved using the resources stored in the cache. If the resource is not in the cache, a suitable error, such as ERROR_FILE_NOT_FOUND, is returned.
INTERNET_FLAG_RELOADForces the function to retrieve the requested resource directly from the Internet. The information that is downloaded is stored in the cache.
INTERNET_FLAG_RESYNCHRONIZECauses an application to perform a conditional download of the resource from the Internet. If the version stored in the cache is current, the information is downloaded from the cache. Otherwise, the information is reloaded from the server.

Persistent Caching Functions

Clients that need persistent caching services use the persistent caching functions to allow their applications to save data in the local file system for subsequent use, such as in situations where a low-bandwidth link limits access to the data, or the access is not available at all. The calling program that inserts data into the persistent cache assigns a source name that is used to perform operations, including retrieving, setting and getting some properties, and deleting the data.

The Win32 Internet function protocols use the cache functions to provide persistent caching and offline browsing. Unless the INTERNET_FLAG_NO_CACHE_WRITE flag explicitly specifies no caching, Win32 Internet functions cache all data downloaded from the network. The responses to POST data are not cached.

Using the Persistent URL Cache Functions

The following persistent URL cache functions allow an application to access and manipulate information stored in the cache.
Function Description
CommitUrlCacheEntry Caches data in the specified file in the cache storage and associates it with the given URL.
CreateUrlCacheEntry Allocates the requested cache storage and creates a local file name for saving the cache entry corresponding to the source name.
CreateUrlCacheGroup Generates a cache group identification.
DeleteUrlCacheEntry Removes the file associated with the source name from the cache, if the file exists.
DeleteUrlCacheGroup Releases a GROUPID and any associated state in the cache index file.
FindCloseUrlCache Closes the specified enumeration handle.
FindFirstUrlCacheEntry Begins the enumeration of the cache.
FindFirstUrlCacheEntryEx Begins a filtered enumeration of the cache.
FindNextUrlCacheEntry Retrieves the next entry in the cache.
FindNextUrlCacheEntryEx Retrieves the next entry in a filtered cache enumeration.
GetUrlCacheEntryInfo Retrieves information about a cache entry.
GetUrlCacheEntryInfoEx Searches for the URL after translating any cached redirections that would be applied in offline mode by HttpSendRequest.
ReadUrlCacheEntryStream Reads the cached data from a stream that has been opened using RetrieveUrlCacheEntryStream.
RetrieveUrlCacheEntryFile Retrieves a cache entry from the cache in the form of a file.
RetrieveUrlCacheEntryStream Provides the most efficient and implementation-independent way of accessing the cache data.
SetUrlCacheEntryGroup Adds or removes entries from a cache group.
SetUrlCacheEntryInfo Sets the specified members of the INTERNET_CACHE_ENTRY_INFO structure.
UnlockUrlCacheEntryFile Unlocks the cache entry that was locked while the file was retrieved for use from the cache.
UnlockUrlCacheEntryStream Closes the stream that has been retrieved using RetrieveUrlCacheEntryStream.

Enumerating the cache

The FindFirstUrlCacheEntry and FindNextUrlCacheEntry functions enumerate the information stored in the cache. FindFirstUrlCacheEntry starts the enumeration by taking a search pattern, a buffer, and a buffer size to create a handle and return the first cache entry. FindNextUrlCacheEntry takes the handle created by FindFirstUrlCacheEntry, a buffer, and a buffer size to return the next cache entry.

Both functions store an INTERNET_CACHE_ENTRY_INFO structure in the buffer. The size of this structure varies for each entry. If the buffer size passed to either function is insufficient, the function fails and GetLastError returns ERROR_INSUFFICIENT_BUFFER. The buffer size variable contains the buffer size that was needed to retrieve that cache entry. A buffer of the size indicated by the buffer size variable should be allocated, and the function should be called again with the new buffer.

The INTERNET_CACHE_ENTRY_INFO structure contains the structure size; URL of the cached information; local file name; cache entry type; use count; hit rate; size; last modified, expire, last access, and last synchronized times; header information and header information size; and file extension.

FindFirstUrlCacheEntry takes a search pattern, a buffer that stores the INTERNET_CACHE_ENTRY_INFO structure, and the buffer size. Currently, only the default search pattern, which returns all cache entries, is implemented.

After the cache is enumerated, the application should call FindCloseUrlCache to close the cache enumeration handle.

The following example displays the URL of each entry in the cache into a list box, IDC_CacheList. It uses MAX_CACHE_ENTRY_INFO_SIZE to initially allocate a buffer, since the Microsoft Internet Explorer 3.0 version of WinInet did not enumerate the cache properly otherwise. Internet Explorer 4.0 does enumerate the cache properly and there is no cache size limit. All applications that run on computers with the Internet Explorer 4.0 version of WinInet must allocate a buffer of the size required. For more information, see Appendix B: Using Buffers. 

int WINAPI EnumerateCacheOld(HWND hX)
{
    DWORD dwEntrySize;
    LPINTERNET_CACHE_ENTRY_INFO lpCacheEntry;
    DWORD MAX_CACHE_ENTRY_INFO_SIZE=4096;
    HANDLE hCacheDir;
    int nCount=0;

    SendDlgItemMessage(hX,IDC_CacheList,LB_RESETCONTENT,0,0);


    
    SetCursor(LoadCursor(NULL,IDC_WAIT));

    dwEntrySize = MAX_CACHE_ENTRY_INFO_SIZE;
    lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) new char[dwEntrySize];
    lpCacheEntry->dwStructSize = dwEntrySize;

again:

    if (!(hCacheDir = FindFirstUrlCacheEntry(NULL,lpCacheEntry,&dwEntrySize)))
    {
        delete[]lpCacheEntry;
        switch(GetLastError())
        {
            case ERROR_NO_MORE_ITEMS: 
                char tempout[80];
                sprintf(tempout,"The number of cache entries = %d \n",nCount);
                MessageBox(hX,tempout,"Cache Enumeration",MB_OK);
                FindCloseUrlCache(hCacheDir);
                SetCursor(LoadCursor(NULL,IDC_ARROW));
                return TRUE;
                break;
                        case ERROR_INSUFFICIENT_BUFFER:
                                lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) 
                                                new char[dwEntrySize];
                                lpCacheEntry->dwStructSize = dwEntrySize;
                                goto again;
                                break;
                        default:
                                ErrorOut(hX,GetLastError(),"FindNextUrlCacheEntry Init");
                                FindCloseUrlCache(hCacheDir);
                                SetCursor(LoadCursor(NULL,IDC_ARROW));
                                return FALSE;
                }
        }

SendDlgItemMessage(hX,IDC_CacheList,LB_ADDSTRING,
                                0,(LPARAM)(lpCacheEntry->lpszSourceUrlName));
        nCount++;
        delete (lpCacheEntry);

        do 
        {
                dwEntrySize = MAX_CACHE_ENTRY_INFO_SIZE;
                lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) new char[dwEntrySize];
                lpCacheEntry->dwStructSize = dwEntrySize;

retry:
                if (!FindNextUrlCacheEntry(hCacheDir,lpCacheEntry, &dwEntrySize))
                {
                        delete[]lpCacheEntry;
                        switch(GetLastError())
                        {
                                case ERROR_NO_MORE_ITEMS: 
                                        char tempout[80];
                                        sprintf(tempout,"The number of cache entries = 
                                                %d \n",nCount);
                                        MessageBox(hX,tempout,"Cache Enumeration",MB_OK);
                                        FindCloseUrlCache(hCacheDir);
                                        return TRUE;
                                        break;
                                case ERROR_INSUFFICIENT_BUFFER:
                                        lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) 
                                                        new char[dwEntrySize];
                                        lpCacheEntry->dwStructSize = dwEntrySize;
                                        goto retry;
                                        break;
                                default:
                                        ErrorOut(hX,GetLastError(),"FindNextUrlCacheEntry Init");
                                        FindCloseUrlCache(hCacheDir);
                                        return FALSE;
                        }
                }

SendDlgItemMessage(hX,IDC_CacheList,LB_ADDSTRING,
                                        0,(LPARAM)(lpCacheEntry->lpszSourceUrlName));
                nCount++;
                delete[] lpCacheEntry;

        
        }
        while (TRUE);

        SetCursor(LoadCursor(NULL,IDC_ARROW));
        return TRUE;
        
}

Retrieving cache entry information

The GetUrlCacheEntryInfo function allows you to retrieve the INTERNET_CACHE_ENTRY_INFO structure for the specified URL. This structure contains the structure size; URL of the cached information; local file name; cache entry type; use count; hit rate; size; last modified, expire, last access, and last synchronized times; header information and header information size; and file extension.

GetUrlCacheEntryInfo accepts a URL, a buffer for an INTERNET_CACHE_ENTRY_INFO structure, and the buffer size. If the URL is found, the information is copied into the buffer. Otherwise, the function fails and GetLastError returns ERROR_FILE_NOT_FOUND. If the buffer size is insufficient to store the cache entry information, the function fails and GetLastError returns ERROR_INSUFFICIENT_BUFFER. The size required to retrieve the information is stored in the buffer size variable.

GetUrlCacheEntryInfo does not do any URL parsing, so a URL containing an anchor (#) will not be found in the cache, even if the resource is cached. For example, if the URL, http://example.com/example.htm#sample, was passed, the function would return ERROR_FILE_NOT_FOUND even if http://example.com/example.htm is in the cache.

The following example retrieves the cache entry information for the specified URL. The function then displays the header information in the IDC_CacheDump edit box. 

int WINAPI GetCacheEntryInfo(HWND hX,LPSTR lpszUrl)
{
        DWORD dwEntrySize=0;
        LPINTERNET_CACHE_ENTRY_INFO lpCacheEntry;

        char strTemp[80];
        DWORD dwTemp;

        SetCursor(LoadCursor(NULL,IDC_WAIT));
        if (!GetUrlCacheEntryInfo(lpszUrl,NULL,&dwEntrySize))
        {
                if (GetLastError()!=ERROR_INSUFFICIENT_BUFFER)
                {
                        ErrorOut(hX,GetLastError(),"GetUrlCacheEntryInfo");
                        SetCursor(LoadCursor(NULL,IDC_ARROW));
                        return FALSE;
                }
                else
                        lpCacheEntry = (LPINTERNET_CACHE_ENTRY_INFO) 
                                        new char[dwEntrySize];
        }
        else
                return FALSE;    //should not be successful w/ NULL buffer and 0 size.

        if (!GetUrlCacheEntryInfo(lpszUrl,lpCacheEntry,&dwEntrySize))
        {
                ErrorOut(hX,GetLastError(),"GetUrlCacheEntryInfo");
                SetCursor(LoadCursor(NULL,IDC_ARROW));
                return FALSE;
        }
        else
        {
                if ((lpCacheEntry->dwHeaderInfoSize)!=0)
                {
                        LPSTR(lpCacheEntry->lpHeaderInfo)[lpCacheEntry->dwHeaderInfoSize]='\0';
                        SetDlgItemText(hX,IDC_Headers,LPSTR(lpCacheEntry->lpHeaderInfo));
                }
                else
                {
                        SetDlgItemText(hX,IDC_Headers,"None");
                }

SetCursor(LoadCursor(NULL,IDC_ARROW));
                return TRUE;
        }
        
}

Creating a cache entry

An application uses the CreateUrlCacheEntry and CommitUrlCacheEntry functions to create a cache entry.

CreateUrlCacheEntry accepts the URL, expected file size, and file extension. The function then creates a local file name for saving the cache entry corresponding to the URL and file extension.

Using the local file name, write the data into the local file using standard C/C++ functions or Win32 functions. After the data has been written to the local file, the application should call CommitUrlCacheEntry.

CommitUrlCacheEntry accepts the URL; local file name; expire and last modified times; cache entry type; header information and header information size; and file extension. The function then caches data in the file specified in the cache storage and associates it with the given URL.

The following example uses the local file name, created by a previous call to CreateUrlCacheEntry, stored in the text box, IDC_LocalFile, to store the text from the text box, IDC_CacheDump, in the cache entry. After the data has been written to the file using fopen, fprintf, and fclose, the entry is committed using CommitUrlCacheEntry. 

int WINAPI CommitEntry(HWND hX)
{
        LPSTR lpszUrl, lpszExt, lpszFileName;
        LPSTR lpszData,lpszSize;
        DWORD dwSize;
        DWORD dwEntryType=0;
        FILE *lpfCacheEntry;
        LPFILETIME lpdtmExpire, lpdtmLastModified;
        LPSYSTEMTIME lpdtmSysTime;

        if(SendDlgItemMessage(hX,IDC_RBNormal,BM_GETCHECK,0,0))
                dwEntryType = dwEntryType + NORMAL_CACHE_ENTRY;
        else
                if(SendDlgItemMessage(hX,IDC_RBStable, BM_GETCHECK,0,0))
                        dwEntryType = dwEntryType + STABLE_CACHE_ENTRY;
                else
                        if(SendDlgItemMessage(hX,IDC_RBSticky, BM_GETCHECK,0,0))
                                dwEntryType = dwEntryType + STICKY_CACHE_ENTRY;
                        else
                                if(SendDlgItemMessage(hX,IDC_RBSparse, BM_GETCHECK,0,0))
                                        dwEntryType = dwEntryType + SPARSE_CACHE_ENTRY;

        if(SendDlgItemMessage(hX,IDC_RBOCX, BM_GETCHECK,0,0))
                dwEntryType = dwEntryType + OCX_CACHE_ENTRY;
        else
                if(SendDlgItemMessage(hX,IDC_RBCookie, BM_GETCHECK,0,0))
                        dwEntryType = dwEntryType + COOKIE_CACHE_ENTRY;
                else
                        if(SendDlgItemMessage(hX,IDC_RBUrl, BM_GETCHECK,0,0))
                                dwEntryType = dwEntryType + URLHISTORY_CACHE_ENTRY;

        if(SendDlgItemMessage(hX,IDC_RBNone, BM_GETCHECK,0,0))
        {
                dwEntryType=0;
        }
        

        lpdtmSysTime = new SYSTEMTIME;
        lpdtmExpire = new FILETIME;
        lpdtmLastModified = new FILETIME;

        GetLocalTime(lpdtmSysTime);
        SystemTimeToFileTime(lpdtmSysTime,lpdtmExpire);
        SystemTimeToFileTime(lpdtmSysTime,lpdtmLastModified);
        delete(lpdtmSysTime);

        lpszUrl = new char[MAX_PATH];
        lpszFileName = new char[MAX_PATH];
        lpszExt = new char[5];
        lpszSize = new char[10];

        GetDlgItemText(hX,IDC_SourceURL,lpszUrl,MAX_PATH);
        GetDlgItemText(hX,IDC_LocalFile,lpszFileName,MAX_PATH);
        GetDlgItemText(hX,IDC_FileExt,lpszExt,5);

        GetDlgItemText(hX,IDC_SizeLow,lpszSize,5);
        dwSize = (DWORD)atol(lpszSize);
        delete(lpszSize);

        if (dwSize==0)
        {
                if((MessageBox(hX,"Incorrect File Size.\nUsing 8000 characters, Okay?\n",
                        "Commit Entry",MB_YESNO))==IDYES)
                {
                        dwSize = 8000;
                }
                else
                {
                        return FALSE;
                }
        }

        lpszData = new char[dwSize];
        GetDlgItemText(hX,IDC_CacheDump,lpszData,dwSize);
        
        lpfCacheEntry = fopen(lpszFileName,"w");
        fprintf(lpfCacheEntry,"%s",lpszData);
        fclose(lpfCacheEntry);
        delete(lpszData);

        if (!CommitUrlCacheEntry(lpszUrl, lpszFileName, *lpdtmExpire,
                *lpdtmLastModified, dwEntryType,NULL,0,lpszExt,0))
        {
                ErrorOut(hX,GetLastError(),"Commit Cache Entry");
                delete(lpszUrl);
                delete(lpszFileName);
                delete(lpszExt);
                delete(lpdtmExpire);
                delete(lpdtmLastModified);
                return FALSE;
        }
        else
        {
                delete(lpszUrl);
                delete(lpszFileName);
                delete(lpszExt);
                delete(lpdtmExpire);
                delete(lpdtmLastModified);
                return TRUE;
        }
}

Deleting a cache entry

The DeleteUrlCacheEntry function takes a URL and removes the cache file associated with it. If the cache file does not exist, the function fails and GetLastError returns ERROR_FILE_NOT_FOUND. If the cache file is currently locked or in use, the function fails and GetLastError returns ERROR_ACCESS_DENIED, and the file will be deleted when unlocked.

Retrieving a cache entry stream

RetrieveUrlCacheEntryStream, ReadUrlCacheEntryStream, and UnlockUrlCacheEntryStream retrieve the resources stored in the cache.

RetrieveUrlCacheEntryStream accepts a URL, a buffer that stores the INTERNET_CACHE_ENTRY_INFO structure, the buffer size, and a Boolean value to determine if random reads are allowed. If the cache file is found, the function then creates a handle to the file.

RetrieveUrlCacheEntryStream does not do any URL parsing, so a URL containing an anchor (#) will not be found in the cache, even if the resource is cached. For example, if the URL, http://example.com/example.htm#sample, was passed, the function would return ERROR_FILE_NOT_FOUND even if http://example.com/example.htm is in the cache.

ReadUrlCacheEntryStream uses the handle created by RetrieveUrlCacheEntryStream, a file offset, a buffer, and a buffer size variable. If the buffer size is insufficient to hold the amount of data available, the function fails and GetLastError returns ERROR_INSUFFICIENT_BUFFER. The buffer size variable is set to the size necessary to download the resource.

After the cache file is retrieved, the application must call UnlockUrlCacheEntryStream to close the handle that was created by RetrieveUrlCacheEntryStream.

The following example retrieves the information stored in the cache entry for the specified URL. The function displays text resources in the IDC_CacheDump edit box. Other resources are not displayed. 

int WINAPI RetrieveStream(HWND hX, LPSTR lpszUrl)
{
    LPINTERNET_CACHE_ENTRY_INFO lpCacheEntry;
    DWORD dwEntrySize=0,dwStreamSize=0;
    HANDLE hStream;
    LPSTR lpszOut;

    RetrieveUrlCacheEntryStream(lpszUrl,NULL, &dwEntrySize, TRUE, 0);
    lpCacheEntry = LPINTERNET_CACHE_ENTRY_INFO(new char[dwEntrySize]);
    if (!(hStream = RetrieveUrlCacheEntryStream(lpszUrl,lpCacheEntry, 
        &dwEntrySize, TRUE, 0)))
    {
        ErrorOut(hX,GetLastError(),"RetrieveUrlCacheEntryStream");
        return FALSE;
    }
    else
    {
        dwStreamSize = lpCacheEntry->dwSizeLow;
        lpszOut = new char[dwStreamSize];
        if (!ReadUrlCacheEntryStream(hStream,0,LPVOID(lpszOut),&dwStreamSize,
            0))
        {
            ErrorOut(hX,GetLastError(),"ReadUrlCacheEntryStream");
            return FALSE;
        }
        else
        {
            lpszOut[dwStreamSize]='\0';
            SetDlgItemText(hX,IDC_CacheDump,lpszOut);
            return TRUE;
        }
    }

    delete (lpCacheEntry);
    delete (lpszOut);
    
    if (!UnlockUrlCacheEntryStream(hStream,0))
    {
        ErrorOut(hX,GetLastError(),"UnlockUrlCacheEntryStream");
        return FALSE;
    }
    else
        return TRUE;
}

Retrieving cache entry files

For applications that require the file name of a resource in order to launch, the Win32 Internet API provides the RetrieveUrlCacheEntryFile and UnlockUrlCacheEntryFile functions. Applications that do not require the file name should use the RetrieveUrlCacheEntryStream, ReadUrlCacheEntryStream, and UnlockUrlCacheEntryStream functions to retrieve the information in the cache.

RetrieveUrlCacheEntryStream does not do any URL parsing, so a URL containing an anchor (#) will not be found in the cache, even if the resource is cached. For example, if the URL, http://example.com/example.htm#sample, was passed, the function would return ERROR_FILE_NOT_FOUND even if http://example.com/example.htm is in the cache.

RetrieveUrlCacheEntryFile accepts a URL, a buffer that stores the INTERNET_CACHE_ENTRY_INFO structure, and the buffer size. The function is retrieved and locked for the caller.

After the information in the file has been used, the application should call UnlockUrlCacheEntryFile to unlock the file.

Cache groups

Support for cache groups has been added to the Win32 Internet functions. To create a cache group, the CreateUrlCacheGroup function must be called to generate a GROUPID for the cache group. Entries can be added to the cache group by supplying the cache entry's URL and the INTERNET_CACHE_GROUP_ADD flag to the SetUrlCacheEntryGroup function. To remove a cache entry from a group, pass the cache entry's URL and the INTERNET_CACHE_GROUP_REMOVE flag to SetUrlCacheEntryGroup.

The FindFirstUrlCacheEntryEx and FindNextUrlCacheEntryEx functions can be used to enumerate the entries in a specified cache group. After the enumeration is complete, the function should call FindCloseUrlCache.

Handling Structures with Variable Size Information

The cache can contain variable size information for each URL stored. This is reflected in the INTERNET_CACHE_ENTRY_INFO structure. When the cache functions return this structure, they create a buffer that is always the size of INTERNET_CACHE_ENTRY_INFO plus any variable size information. If a pointer member is not NULL, it points to the memory area immediately after the structure. While copying the returned buffer from a function into another buffer, the pointer members should be fixed to point to the appropriate place in the new buffer, as the following example shows. 

lpDstCEInfo->lpszSourceUrlName = 
    (LPINTERNET_CACHE_ENTRY_INFO) ((LPBYTE) lpSrcCEInfo + 
        ((DWORD) (lpOldCEInfo->lpszSourceUrlName) - (DWORD) lpOldCEInfo))

Some cache functions fail with the ERROR_INSUFFICIENT_BUFFER error message if you specify a buffer that is too small to contain the cache-entry information retrieved by the function. In this case, the function also returns the required size of the buffer. You can then allocate a buffer of the appropriate size and call the function again.

Up Top of Page
© 1997 Microsoft Corporation. All rights reserved. Terms of Use.