netty解决粘包半包问题
2019-08-16 09:43:34来源：博客园 阅读 ()
netty解决粘包半包问题
前言：开发者用到TCP/IP交互时，偶尔会遇到粘包或者半包的数据，这种情况有时会对我们的程序造成严重的影响，netty框架为解决这种问题提供了若干框架
1. LineBasedFrameDecoder：通过在包尾添加回车换行符 \r\n 来区分整包消息。
说明：LineBasedFrameDecoder 是从 ByteBuf 的可读字节中找到 \n 或者 \r\n，找到之后就以此为结束，然后将当前读取到的数据组成一行。
使用方法：ch.pipline().addLast(new LineBasedFrameDecoder(1024));//1024是设置每一行的最大长度，如果读到最大长度还没有结束符，会抛出异常，结束当前读取到的数据
2. DelimiterBasedFrameDecoder：通过特殊字符作为分隔符来区分整包消息
说明：客户端和服务端协议中有一致的特殊字符，来代表每一包的结束
如：客户端发送3条消息："你好你好你好$_"
"hellohellohello$_"
"赞赞$_"
服务端为了防止分包粘包，使用方法如下：
ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());
ch.pipline().addLast(new DelimiterBasedFrameDecoder(1024, delimiter));//1024设置每一行的最大长度
3.FixedLengthFrameDecoder：用于定常消息，消息固定长度，不够空格补全，发送方和接收方遵循相同规定
使用方法：ch.pipline().addLast(new FixedLengthFrameDecoder(1024));
以上几种方法在实际项目中，存在一定的局限性，协议中可能不会有\n这种字符，或者也没有第二种方法提到的特殊字符，第三种方法一般不实用，很少有项目的每条协议字段都是一样的长度。
那就介绍一种科一解决不定长协议的分包粘包方法：LengthFieldBasedFrameDecoder
先看一下LengthFieldBasedFrameDecoder源码中一个构造函数：
public LengthFieldBasedFrameDecoder(
int maxFrameLength,
int lengthFieldOffset, int lengthFieldLength,
int lengthAdjustment, int initialBytesToStrip) {
this(
maxFrameLength,
lengthFieldOffset, lengthFieldLength, lengthAdjustment,
initialBytesToStrip, true);
}
其中有5个参数的定义我用一个协议数据举例
maxFrameLength：单个包的最大长度，根据实际环境定义
lengthFieldOffset：表示数据长度字段开始的偏移量，上图中，表示数据长度的字段前边还有协议头、版本号、日期等字段，一共占了12字节，所以按照这个协议，这个参数填写12
lengthFieldOffset：数据长度字段所占的字节数，上图中协议长度占用2个字节，这个参数写2
lengthAdjustment：lengthAdjustment +数据长度取值 = 数据长度字段之后剩下包的字节数，上图中，除了协议长度和数据内容，后边还有协议尾和CRC认证，占用6个字节，这个参数写6
initialBytesToStrip：从整个包开始，向后忽略的字节数（我写成0）
按照以上协议，我的使用方法就是：ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024,12,2,6,0));
这句代码加在自定义Hander的前边，否则无效。这样就解决了粘包和分包的问题
但是我现在遇见的问题是：我的协议数据长度字段和数据内容实际长度可能是不对应的
我的协议是如果长度字段不是8的倍数，要在数据后补0 ，例如：
协议长度字段是0046 46不是8的倍数 要补两个0 所以导致以上所有方法都不适合我，因为我不知道长度究竟会是几，究竟会补几个0，于是我自定义LengthFieldBasedFrameDecoder解决这个问题
源码中LengthFieldBasedFrameDecoder继承的是ByteToMessageDecoder（以上所有方法都是继承这个类）
解决方案：自己创建一个类，继承ByteToMessageDecoder，将LengthFieldBasedFrameDecoder
public class SelfDefineEncodeHandler extends ByteToMessageDecoder {
private final ByteOrder byteOrder;
private final int maxFrameLength;
private final int lengthFieldOffset;
private final int lengthFieldLength;
private final int lengthFieldEndOffset;
private final int lengthAdjustment;
private final int initialBytesToStrip;
private final boolean failFast;
private boolean discardingTooLongFrame;
private long tooLongFrameLength;
private long bytesToDiscard;
/**
* Creates a new instance.
*
* @param maxFrameLength
* the maximum length of the frame. If the length of the frame is
* greater than this value, {@link TooLongFrameException} will be
* thrown.
* @param lengthFieldOffset
* the offset of the length field
* @param lengthFieldLength
* the length of the length field
*/
public SelfDefineEncodeHandler(
int maxFrameLength,
int lengthFieldOffset, int lengthFieldLength) {
this(maxFrameLength, lengthFieldOffset, lengthFieldLength, 0, 0);
}
/**
* Creates a new instance.
*
* @param maxFrameLength
* the maximum length of the frame. If the length of the frame is
* greater than this value, {@link TooLongFrameException} will be
* thrown.
* @param lengthFieldOffset
* the offset of the length field
* @param lengthFieldLength
* the length of the length field
* @param lengthAdjustment
* the compensation value to add to the value of the length field
* @param initialBytesToStrip
* the number of first bytes to strip out from the decoded frame
*/
public SelfDefineEncodeHandler(
int maxFrameLength,
int lengthFieldOffset, int lengthFieldLength,
int lengthAdjustment, int initialBytesToStrip) {
this(
maxFrameLength,
lengthFieldOffset, lengthFieldLength, lengthAdjustment,
initialBytesToStrip, true);
}
/**
* Creates a new instance.
*
* @param maxFrameLength
* the maximum length of the frame. If the length of the frame is
* greater than this value, {@link TooLongFrameException} will be
* thrown.
* @param lengthFieldOffset
* the offset of the length field
* @param lengthFieldLength
* the length of the length field
* @param lengthAdjustment
* the compensation value to add to the value of the length field
* @param initialBytesToStrip
* the number of first bytes to strip out from the decoded frame
* @param failFast
* If <tt>true</tt>, a {@link TooLongFrameException} is thrown as
* soon as the decoder notices the length of the frame will exceed
* <tt>maxFrameLength</tt> regardless of whether the entire frame
* has been read. If <tt>false</tt>, a {@link TooLongFrameException}
* is thrown after the entire frame that exceeds <tt>maxFrameLength</tt>
* has been read.
*/
public SelfDefineEncodeHandler(
int maxFrameLength, int lengthFieldOffset, int lengthFieldLength,
int lengthAdjustment, int initialBytesToStrip, boolean failFast) {
this(
ByteOrder.BIG_ENDIAN, maxFrameLength, lengthFieldOffset, lengthFieldLength,
lengthAdjustment, initialBytesToStrip, failFast);
}
/**
* Creates a new instance.
*
* @param byteOrder
* the {@link ByteOrder} of the length field
* @param maxFrameLength
* the maximum length of the frame. If the length of the frame is
* greater than this value, {@link TooLongFrameException} will be
* thrown.
* @param lengthFieldOffset
* the offset of the length field
* @param lengthFieldLength
* the length of the length field
* @param lengthAdjustment
* the compensation value to add to the value of the length field
* @param initialBytesToStrip
* the number of first bytes to strip out from the decoded frame
* @param failFast
* If <tt>true</tt>, a {@link TooLongFrameException} is thrown as
* soon as the decoder notices the length of the frame will exceed
* <tt>maxFrameLength</tt> regardless of whether the entire frame
* has been read. If <tt>false</tt>, a {@link TooLongFrameException}
* is thrown after the entire frame that exceeds <tt>maxFrameLength</tt>
* has been read.
*/
public SelfDefineEncodeHandler(
ByteOrder byteOrder, int maxFrameLength, int lengthFieldOffset, int lengthFieldLength,
int lengthAdjustment, int initialBytesToStrip, boolean failFast) {
if (byteOrder == null) {
throw new NullPointerException("byteOrder");
}
if (maxFrameLength <= 0) {
throw new IllegalArgumentException(
"maxFrameLength must be a positive integer: " +
maxFrameLength);
}
if (lengthFieldOffset < 0) {
throw new IllegalArgumentException(
"lengthFieldOffset must be a non-negative integer: " +
lengthFieldOffset);
}
if (initialBytesToStrip < 0) {
throw new IllegalArgumentException(
"initialBytesToStrip must be a non-negative integer: " +
initialBytesToStrip);
}
if (lengthFieldOffset > maxFrameLength - lengthFieldLength) {
throw new IllegalArgumentException(
"maxFrameLength (" + maxFrameLength + ") " +
"must be equal to or greater than " +
"lengthFieldOffset (" + lengthFieldOffset + ") + " +
"lengthFieldLength (" + lengthFieldLength + ").");
}
this.byteOrder = byteOrder;
this.maxFrameLength = maxFrameLength;
this.lengthFieldOffset = lengthFieldOffset;
this.lengthFieldLength = lengthFieldLength;
this.lengthAdjustment = lengthAdjustment;
lengthFieldEndOffset = lengthFieldOffset + lengthFieldLength;
this.initialBytesToStrip = initialBytesToStrip;
this.failFast = failFast;
}
@Override
protected final void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
Object decoded = decode(ctx, in);
if (decoded != null) {
out.add(decoded);
}
}
/**
* Create a frame out of the {@link ByteBuf} and return it.
*
* @param ctx the {@link ChannelHandlerContext} which this {@link ByteToMessageDecoder} belongs to
* @param in the {@link ByteBuf} from which to read data
* @return frame the {@link ByteBuf} which represent the frame or {@code null} if no frame could
* be created.
*/
protected Object decode(ChannelHandlerContext ctx, ByteBuf in) throws Exception {
if (discardingTooLongFrame) {
long bytesToDiscard = this.bytesToDiscard;
int localBytesToDiscard = (int) Math.min(bytesToDiscard, in.readableBytes());
in.skipBytes(localBytesToDiscard);
bytesToDiscard -= localBytesToDiscard;
this.bytesToDiscard = bytesToDiscard;
failIfNecessary(false);
}
if (in.readableBytes() < lengthFieldEndOffset) {
return null;
}
int actualLengthFieldOffset = in.readerIndex() + lengthFieldOffset;
long frameLength = getUnadjustedFrameLength(in, actualLengthFieldOffset, lengthFieldLength, byteOrder);
System.out.println("---------------"+frameLength);
if (frameLength < 0) {
in.skipBytes(lengthFieldEndOffset);
throw new CorruptedFrameException(
"negative pre-adjustment length field: " + frameLength);
}
frameLength += lengthAdjustment + lengthFieldEndOffset;
if (frameLength < lengthFieldEndOffset) {
in.skipBytes(lengthFieldEndOffset);
throw new CorruptedFrameException(
"Adjusted frame length (" + frameLength + ") is less " +
"than lengthFieldEndOffset: " + lengthFieldEndOffset);
}
if (frameLength > maxFrameLength) {
long discard = frameLength - in.readableBytes();
tooLongFrameLength = frameLength;
if (discard < 0) {
// buffer contains more bytes then the frameLength so we can discard all now
in.skipBytes((int) frameLength);
} else {
// Enter the discard mode and discard everything received so far.
discardingTooLongFrame = true;
bytesToDiscard = discard;
in.skipBytes(in.readableBytes());
}
failIfNecessary(true);
return null;
}
// never overflows because it's less than maxFrameLength
int frameLengthInt = (int) frameLength;
if (in.readableBytes() < frameLengthInt) {
return null;
}
if (initialBytesToStrip > frameLengthInt) {
in.skipBytes(frameLengthInt);
throw new CorruptedFrameException(
"Adjusted frame length (" + frameLength + ") is less " +
"than initialBytesToStrip: " + initialBytesToStrip);
}
in.skipBytes(initialBytesToStrip);
// extract frame
int readerIndex = in.readerIndex();
int actualFrameLength = frameLengthInt - initialBytesToStrip;
ByteBuf frame = extractFrame(ctx, in, readerIndex, actualFrameLength);
in.readerIndex(readerIndex + actualFrameLength);
return frame;
}
/**
* Decodes the specified region of the buffer into an unadjusted frame length. The default implementation is
* capable of decoding the specified region into an unsigned 8/16/24/32/64 bit integer. Override this method to
* decode the length field encoded differently. Note that this method must not modify the state of the specified
* buffer (e.g. {@code readerIndex}, {@code writerIndex}, and the content of the buffer.)
*
* @throws DecoderException if failed to decode the specified region
*/
protected long getUnadjustedFrameLength(ByteBuf buf, int offset, int length, ByteOrder order) {
buf = buf.order(order);
long frameLength;
switch (length) {
case 1:
frameLength = buf.getUnsignedByte(offset);
break;
case 2:
frameLength = buf.getUnsignedShort(offset);
break;
case 3:
frameLength = buf.getUnsignedMedium(offset);
break;
case 4:
frameLength = buf.getUnsignedInt(offset);
break;
case 8:
frameLength = buf.getLong(offset);
break;
default:
throw new DecoderException(
"unsupported lengthFieldLength: " + lengthFieldLength + " (expected: 1, 2, 3, 4, or 8)");
}
return frameLength+8-(frameLength%8);
}
private void failIfNecessary(boolean firstDetectionOfTooLongFrame) {
if (bytesToDiscard == 0) {
// Reset to the initial state and tell the handlers that
// the frame was too large.
long tooLongFrameLength = this.tooLongFrameLength;
this.tooLongFrameLength = 0;
discardingTooLongFrame = false;
if (!failFast ||
failFast && firstDetectionOfTooLongFrame) {
fail(tooLongFrameLength);
}
} else {
// Keep discarding and notify handlers if necessary.
if (failFast && firstDetectionOfTooLongFrame) {
fail(tooLongFrameLength);
}
}
}
/**
* Extract the sub-region of the specified buffer.
* <p>
* If you are sure that the frame and its content are not accessed after
* the current {@link #decode(ChannelHandlerContext, ByteBuf)}
* call returns, you can even avoid memory copy by returning the sliced
* sub-region (i.e. <tt>return buffer.slice(index, length)</tt>).
* It's often useful when you convert the extracted frame into an object.
* Refer to the source code of {@link ObjectDecoder} to see how this method
* is overridden to avoid memory copy.
*/
protected ByteBuf extractFrame(ChannelHandlerContext ctx, ByteBuf buffer, int index, int length) {
return buffer.retainedSlice(index, length);
}
private void fail(long frameLength) {
if (frameLength > 0) {
throw new TooLongFrameException(
"Adjusted frame length exceeds " + maxFrameLength +
": " + frameLength + " - discarded");
} else {
throw new TooLongFrameException(
"Adjusted frame length exceeds " + maxFrameLength +
" - discarding");
}
}
}
备忘，参考文章：http://linkedkeeper.com/detail/blog.action?bid=105 里面介绍了更多netty解决粘包分包的方法
原文链接:https://www.cnblogs.com/yunfang/p/11127919.html
如有疑问请与原作者联系
标签：
版权申明：本站文章部分自网络，如有侵权，请联系：west999com@outlook.com
特别注意：本站所有转载文章言论不代表本站观点，本站所提供的摄影照片，插画，设计作品，如需使用，请与原作者联系，版权归原作者所有
相关文章
- 聊聊微服务架构及分布式事务解决方案！ 2020-06-10
- 这可能是目前最透彻的Netty讲解了... 2020-06-08
- 解决IDEA Maven下载依赖包速度慢问题 2020-06-05
- Java生鲜电商平台-生鲜电商中配送订单解决方案(小程序/APP) 2020-05-31
- 一口气说出8种幂等性解决重复提交的方案，面试官懵了！ 2020-05-30
IDC资讯： 主机资讯 注册资讯 托管资讯 vps资讯 网站建设
网站运营： 建站经验 策划盈利 搜索优化 网站推广 免费资源
网络编程： Asp.Net编程 Asp编程 Php编程 Xml编程 Access Mssql Mysql 其它
服务器技术： Web服务器 Ftp服务器 Mail服务器 Dns服务器 安全防护
软件技巧： 其它软件 Word Excel Powerpoint Ghost Vista QQ空间 QQ FlashGet 迅雷
网页制作： FrontPages Dreamweaver Javascript css photoshop fireworks Flash
热门词条
最新资讯
热门关注
热门标签