proxygen: wangle::LengthFieldBasedFrameDecoder Class Reference

#include <LengthFieldBasedFrameDecoder.h>

Inheritance diagram for wangle::LengthFieldBasedFrameDecoder:

Public Member Functions
	LengthFieldBasedFrameDecoder (uint32_t lengthFieldLength=4, uint32_t maxFrameLength=UINT_MAX, uint32_t lengthFieldOffset=0, int32_t lengthAdjustment=0, uint32_t initialBytesToStrip=4, bool networkByteOrder=true)

bool	decode (Context *ctx, folly::IOBufQueue &buf, std::unique_ptr< folly::IOBuf > &result, size_t &) override

Public Member Functions inherited from wangle::ByteToMessageDecoder< M >
virtual bool	decode (Context *ctx, folly::IOBufQueue &buf, M &result, size_t &)=0

void	transportActive (Context *ctx) override

void	transportInactive (Context *ctx) override

void	read (Context *ctx, folly::IOBufQueue &q) override

Public Member Functions inherited from wangle::InboundHandler< folly::IOBufQueue &, M >
	~InboundHandler () override=default

virtual void	read (Context *ctx, folly::IOBufQueue &msg)=0

virtual void	readEOF (Context *ctx)

virtual void	readException (Context *ctx, folly::exception_wrapper e)

virtual void	transportActive (Context *ctx)

virtual void	transportInactive (Context *ctx)

Public Member Functions inherited from wangle::HandlerBase< InboundHandlerContext< M > >
virtual	~HandlerBase ()=default

virtual void	attachPipeline (InboundHandlerContext< M > *)

virtual void	detachPipeline (InboundHandlerContext< M > *)

InboundHandlerContext< M > *	getContext ()

Private Member Functions
uint64_t	getUnadjustedFrameLength (folly::IOBufQueue &buf, int offset, int length, bool networkByteOrder)

Private Attributes
uint32_t	lengthFieldLength_

uint32_t	maxFrameLength_

uint32_t	lengthFieldOffset_

int32_t	lengthAdjustment_

uint32_t	initialBytesToStrip_

bool	networkByteOrder_

uint32_t	lengthFieldEndOffset_

Additional Inherited Members
Public Types inherited from wangle::ByteToMessageDecoder< M >
typedef InboundHandler< folly::IOBufQueue &, M >::Context	Context

Public Types inherited from wangle::InboundHandler< folly::IOBufQueue &, M >
typedef folly::IOBufQueue &	rin

typedef M	rout

typedef folly::Unit	win

typedef folly::Unit	wout

typedef InboundHandlerContext< M >	Context

Static Public Attributes inherited from wangle::InboundHandler< folly::IOBufQueue &, M >
static const HandlerDir	dir

Detailed Description

A decoder that splits the received IOBufs dynamically by the value of the length field in the message. It is particularly useful when you decode a binary message which has an integer header field that represents the length of the message body or the whole message.

LengthFieldBasedFrameDecoder has many configuration parameters so that it can decode any message with a length field, which is often seen in proprietary client-server protocols. Here are some example that will give you the basic idea on which option does what.

2 bytes length field at offset 0, do not strip header

The value of the length field in this example is 12 (0x0C) which represents the length of "HELLO, WORLD". By default, the decoder assumes that the length field represents the number of the bytes that follows the length field. Therefore, it can be decoded with the simplistic parameter combination.

lengthFieldOffset = 0 lengthFieldLength = 2 lengthAdjustment = 0 initialBytesToStrip = 0 (= do not strip header)

2 bytes length field at offset 0, strip header

Because we can get the length of the content by calling ioBuf->computeChainDataLength(), you might want to strip the length field by specifying initialBytesToStrip. In this example, we specified 2, that is same with the length of the length field, to strip the first two bytes.

lengthFieldOffset = 0 lengthFieldLength = 2 lengthAdjustment = 0 initialBytesToStrip = 2 (= the length of the Length field)

2 bytes length field at offset 0, do not strip header, the length field represents the length of the whole message

In most cases, the length field represents the length of the message body only, as shown in the previous examples. However, in some protocols, the length field represents the length of the whole message, including the message header. In such a case, we specify a non-zero lengthAdjustment. Because the length value in this example message is always greater than the body length by 2, we specify -2 as lengthAdjustment for compensation.

lengthFieldOffset = 0 lengthFieldLength = 2 lengthAdjustment = -2 (= the length of the Length field) initialBytesToStrip = 0

3 bytes length field at the end of 5 bytes header, do not strip header

The following message is a simple variation of the first example. An extra header value is prepended to the message. lengthAdjustment is zero again because the decoder always takes the length of the prepended data into account during frame length calculation.

lengthFieldOffset = 2 (= the length of Header 1) lengthFieldLength = 3 lengthAdjustment = 0 initialBytesToStrip = 0

3 bytes length field at the beginning of 5 bytes header, do not strip header

This is an advanced example that shows the case where there is an extra header between the length field and the message body. You have to specify a positive lengthAdjustment so that the decoder counts the extra header into the frame length calculation.

lengthFieldOffset = 0 lengthFieldLength = 3 lengthAdjustment = 2 (= the length of Header 1) initialBytesToStrip = 0

2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field

This is a combination of all the examples above. There are the prepended header before the length field and the extra header after the length field. The prepended header affects the lengthFieldOffset and the extra header affects the lengthAdjustment. We also specified a non-zero initialBytesToStrip to strip the length field and the prepended header from the frame. If you don't want to strip the prepended header, you could specify 0 for initialBytesToSkip.

lengthFieldOffset = 1 (= the length of HDR1) lengthFieldLength = 2 lengthAdjustment = 1 (= the length of HDR2) initialBytesToStrip = 3 (= the length of HDR1 + LEN)

BEFORE DECODE (16 bytes) AFTER DECODE (13 bytes) +---—+-----—+---—+-------------—+ +---—+-------------—+ | HDR1 | Length | HDR2 | Actual Content |--—>| HDR2 | Actual Content | | 0xCA | 0x000C | 0xFE | "HELLO, WORLD" | | 0xFE | "HELLO, WORLD" | +---—+-----—+---—+-------------—+ +---—+-------------—+

2 bytes length field at offset 1 in the middle of 4 bytes header, strip the first header field and the length field, the length field represents the length of the whole message

Let's give another twist to the previous example. The only difference from the previous example is that the length field represents the length of the whole message instead of the message body, just like the third example. We have to count the length of HDR1 and Length into lengthAdjustment. Please note that we don't need to take the length of HDR2 into account because the length field already includes the whole header length.

lengthFieldOffset = 1 lengthFieldLength = 2 lengthAdjustment = -3 (= the length of HDR1 + LEN, negative) initialBytesToStrip = 3

BEFORE DECODE (16 bytes) AFTER DECODE (13 bytes) +---—+-----—+---—+-------------—+ +---—+-------------—+ | HDR1 | Length | HDR2 | Actual Content |--—>| HDR2 | Actual Content | | 0xCA | 0x0010 | 0xFE | "HELLO, WORLD" | | 0xFE | "HELLO, WORLD" | +---—+-----—+---—+-------------—+ +---—+-------------—+

See also: LengthFieldPrepender

Definition at line 183 of file LengthFieldBasedFrameDecoder.h.

Constructor & Destructor Documentation

wangle::LengthFieldBasedFrameDecoder::LengthFieldBasedFrameDecoder	(	uint32_t	lengthFieldLength = `4`,
		uint32_t	maxFrameLength = `UINT_MAX`,
		uint32_t	lengthFieldOffset = `0`,
		int32_t	lengthAdjustment = `0`,
		uint32_t	initialBytesToStrip = `4`,
		bool	networkByteOrder = `true`
	)

explicit

Definition at line 24 of file LengthFieldBasedFrameDecoder.cpp.

     : lengthFieldLength_(lengthFieldLength)
     , maxFrameLength_(maxFrameLength)
     , lengthFieldOffset_(lengthFieldOffset)
     , lengthAdjustment_(lengthAdjustment)
     , initialBytesToStrip_(initialBytesToStrip)
     , networkByteOrder_(networkByteOrder)
     , lengthFieldEndOffset_(lengthFieldOffset + lengthFieldLength) {
   CHECK(maxFrameLength > 0);
   CHECK(lengthFieldOffset <= maxFrameLength - lengthFieldLength);
 }

Member Function Documentation

bool wangle::LengthFieldBasedFrameDecoder::decode	(	Context *	ctx,
		folly::IOBufQueue &	buf,
		std::unique_ptr< folly::IOBuf > &	result,
		size_t &
	)

override

Definition at line 42 of file LengthFieldBasedFrameDecoder.cpp.

References folly::IOBufQueue::chainLength(), getUnadjustedFrameLength(), initialBytesToStrip_, lengthAdjustment_, lengthFieldEndOffset_, lengthFieldLength_, lengthFieldOffset_, maxFrameLength_, networkByteOrder_, folly::IOBufQueue::split(), folly::IOBufQueue::trimStart(), folly::IOBufQueue::trimStartAtMost(), and uint64_t.

                                                    {
   // discarding too long frame
   if (buf.chainLength() < lengthFieldEndOffset_) {
     return false;
   }
 
   uint64_t frameLength = getUnadjustedFrameLength(
     buf, lengthFieldOffset_, lengthFieldLength_, networkByteOrder_);
 
   frameLength += lengthAdjustment_ + lengthFieldEndOffset_;
 
   if (frameLength < lengthFieldEndOffset_) {
     buf.trimStart(lengthFieldEndOffset_);
     ctx->fireReadException(folly::make_exception_wrapper<std::runtime_error>(
                              "Frame too small"));
     return false;
   }
 
   if (frameLength > maxFrameLength_) {
     buf.trimStartAtMost(frameLength);
     ctx->fireReadException(folly::make_exception_wrapper<std::runtime_error>(
                              "Frame larger than " +
                              folly::to<std::string>(maxFrameLength_)));
     return false;
   }
 
   if (buf.chainLength() < frameLength) {
     return false;
   }
 
   if (initialBytesToStrip_ > frameLength) {
     buf.trimStart(frameLength);
     ctx->fireReadException(folly::make_exception_wrapper<std::runtime_error>(
                              "InitialBytesToSkip larger than frame"));
     return false;
   }
 
   buf.trimStart(initialBytesToStrip_);
   int actualFrameLength = frameLength - initialBytesToStrip_;
   result = buf.split(actualFrameLength);
   return true;
 }

uint64_t wangle::LengthFieldBasedFrameDecoder::getUnadjustedFrameLength	(	folly::IOBufQueue &	buf,
		int	offset,
		int	length,
		bool	networkByteOrder
	)

private

Definition at line 88 of file LengthFieldBasedFrameDecoder.cpp.

References c, folly::IOBufQueue::front(), uint16_t, uint32_t, uint64_t, and uint8_t.

Referenced by decode().

                                                                   {
   folly::io::Cursor c(buf.front());
   uint64_t frameLength;
 
   c.skip(offset);
 
   switch(length) {
     case 1:{
       if (networkByteOrder) {
         frameLength = c.readBE<uint8_t>();
       } else {
         frameLength = c.readLE<uint8_t>();
       }
       break;
     }
     case 2:{
       if (networkByteOrder) {
         frameLength = c.readBE<uint16_t>();
       } else {
         frameLength = c.readLE<uint16_t>();
       }
       break;
     }
     case 4:{
       if (networkByteOrder) {
         frameLength = c.readBE<uint32_t>();
       } else {
         frameLength = c.readLE<uint32_t>();
       }
       break;
     }
     case 8:{
       if (networkByteOrder) {
         frameLength = c.readBE<uint64_t>();
       } else {
         frameLength = c.readLE<uint64_t>();
       }
       break;
     }
   }
 
   return frameLength;
 }