LibHTTP

[LibHTTP] is a general purpose code package for UnrealEngine2 (developed by El Muerte). It makes it easier to perform HTTP requests to retrieve data stored on webservers. It takes into account most of the aspects that come into the picture when doing correct HTTP requests.

LibHTTP has the following features

• Support for HTTP version 1.0 and 1.1
• Support for GET/POST/HEAD/TRACE request methods
• Normal and accelerated transfer modes (accelerated mode creates a performance hit)
• Response and Request Header managent
• Cookie management
• Authentication supports (both Basic and Digest methods are supported)
• Support for HTTP proxies
• Gracefull handling of connection timeouts
• Automatic decoding of chunked data
• Automatically follows redirections (creates a redirection history)
• Support for multipart/form-data POST data (prefered form)

Version 2, 3 and 4 have been released under the LesserOpenUnrealModLicense.

Version 1 was relesed under the Lesser GPL.

The documentation on the UnrealWiki has been changed to reflect Version 2.

Classes

• /HttpCache - experimental cache manager
• /HttpCacheObject - cache document, created by /HttpCache
• /HttpCookies - Cookie management class
• /HttpLink - the actual TCP Link to retrieve the data (dummy class)
• /HttpResolveCache - hostname resolve cache
• /HttpSock - HTTP client for retreiving data from webservers
• /HttpUtil - static miscelaneous functions
• NewsFeed - a RSS\RDF Parser

Note: the class documentation on the wiki is not up to date for version 4 yet.

Examples

• /Example - a simple example on how to use this package.
• [ServerExt] - the MutRSS packages uses this library to fetch it's data

Releases

• [version 401]
• [version 350] (UT2004)
• [version 300] (UT2004)
• [version 200] (UT2004)
• [version 200] (UT2003)
• [version 100] (UT2003)

The bleeding edge is always available from my CVS repository.

CVS Root for anonymous access

:pserver:anonymous@el-muerte.student.utwente.nl:/usr/local/cvsroot/UT2004
module name: LibHTTP
tag: v400

Custom builds \ subclassing HttpSock

You are allowed to create custom builds of this package (read the license for requirements of custom builds). If you create a custom build you should use a different name for the package so it can be distinguished from the original released (e.g. LibHTTP_MyBuild). Don't use my current scheme of LibHTTP''version number''.

Also if you are creating a custom build you might want to checkout the code from the CVS (note, leave the tag field empty), so you'll start of with the latest changes.

When you do a custom build (and in the future subclass) make sure you set a value for the EXTENTION variable. This variable is used in the UserAgent string. So instead of

  LibHTTP/400 (UnrealEngine2; build 3323; http://wiki.beyondunreal.com/wiki/LibHTTP )

You will get

  LibHTTP/040 (UnrealEngine2; build 3323; http://wiki.beyondunreal.com/wiki/LibHTTP ; MyExtention )

Version 3 release notes

This version isn't a drop-in replacement for version 2. Although the changes you have to make might be minimal.

• Improved easy of use: get(), post(), head()
• Support for multipart/form-data POST data
• Two different transfer modes: Normal and Fast (tries to download as much data as allowed within a single tick)
• Support for proxy authentication, you get the best performance by setting the right user/pass in the beginning. Otherwise the code using this library will have to do additional processing when the proxy user and pass are not accepted.
• Better support for various authentication methods
• Support for digest authentication (more secure HTTP authentication). When digest is used instead of basic the client has to make 2 requests. With the first request the server will send information needed to construct the response. Basic authentication doesn't have this issue.
• Cookie storage class will automatically be created when bProcCookies or bSendCookies is set to true (and the cookies hasn't been set)

Version 3.5 release notes

• All delegates contains a HttpSock Sender argument
• New function string randString(optional int length, optional coerce string prefix)
• MultiPart divider string is now more unique (prevents potential issues with the content data)
• Empty multipart items are never added
• Made more support functions public
• New utility function string timestampToString(int timestamp, optional string Timezone, optional string format) that will convert a UNIX timestamp to a string (RFC 1123, RFC 1036, RFC 2822, ANSI C asctime formats supported)
• Fixed potentional issues with timestamps and cookies

Version 4 release notes

• Various small bugfixes
• Redone the redirection processing, now with better support for each redirection type
• New variable HttpSock.bRfcCompliantRedirect, set this to false for the old, and bad, redirection handling on 301/302 headers (e.g. transform POST into GET)
• Moved DNS caching to a global object, multiple HttpSock classes now use the same cache data
• Added automatic retrying when authentication data is provided in the url, the prefered method for using authenticated locations is to provide the username and password in the url or set them in the currenturl after the OnRequireAuthorization() was received. Can be disabled by setting HttpSock.bAutoAuthenticate to false
• HttpSock.ClearRequestData() will now automatically clear authentication information
• NewsFeed properly reads CDATA
• NewsFeed description fields are truncated to 512 bytes (because of a bug in the engine). Set NewsFeed.bSizeBugFix to false to disable it.
• URL parsing is now done by HttpUtil and stores it in a new xURL struct, all old references to storage of the location and host have been changed to xURL.
• Fixed a couple of bugs in cookie handler, the first cookie was never properly unset
• Fixed a bug where the last line of data may not be properly added to the result array
• Moved the HttpTest class to a UsUnit test suite
• Fixed HttpSock.GetReturnHeader()
• Added a new class HttpCache that manages cached requests, this is somewhat experimental
• Usernames and passwords on the URL are now properly decoded before they're used.

Discussion

MythOpus:El Muerte, how reliable would LipHTTP be for downloading/saving LOTS of information at a given time?

El Muerte: define lots? LibHTTP is reliable as in, it downloads all the data and stores each line into the result array. So, it's more of an issue how reliable a dynamic array is. And there might be an issue with string length (e.g. >4k strings) but I guess you'll have to test that yourself. Anyway, there is a delegate in HttpSock that'll allow you do process the result data before it's stored in the array (see OnResponseBody).

MythOpus: Well, I did do a few tests and found that it doesn't handle my blog very well lol. I am thinking about starting a new project that will require all players to get lots of data from the server. This data might include usernames and passwords, game stats, current game status, as well as journal entries, poems, stories and, if I can get it to download it properly, pictures. Most players would have to download a good portion of this data when they first connect to the server. At that point they would probably have to login.

El Muerte: Pictures? You are aware that you can't do much with binary data right?

MythOpus: Well, I was just thinking that perhaps I could make a picture system. A player could 'draw' a picture in the game and save it and it would save the picture data to a server and when the player wanted to look at it, it would call up that data later and organize it and then display it. A serious of number to show pixel location/color etc.

MythOpus: I've been messing around a bit with the NewsFeed class and I can't see to figure out how to get the actual content in the RSS out and display it in something like a text field.

El Muerte: check out the code of MutRSS ( [1] and [2] )

---

Category Custom Class