interesting
/
go-ethereum
mirror of https://github.com/ethereum/go-ethereum.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
go-ethereum/rlp/encode_test.go

639 lines
22 KiB
Go
Raw Normal View History

all: update license information
2015-07-06 19:54:22 -05:00
// Copyright 2014 The go-ethereum Authors
all: update license headers to distiguish GPL/LGPL All code outside of cmd/ is licensed as LGPL. The headers now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 11:48:40 -05:00
// This file is part of the go-ethereum library.
all: update license information
2015-07-06 19:54:22 -05:00
//
all: fix license headers one more time I forgot to update one instance of "go-ethereum" in commit 3f047be5a.
2015-07-23 11:35:11 -05:00
// The go-ethereum library is free software: you can redistribute it and/or modify
all: update license information
2015-07-06 19:54:22 -05:00
// it under the terms of the GNU Lesser General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
all: update license headers to distiguish GPL/LGPL All code outside of cmd/ is licensed as LGPL. The headers now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 11:48:40 -05:00
// The go-ethereum library is distributed in the hope that it will be useful,
all: update license information
2015-07-06 19:54:22 -05:00
// but WITHOUT ANY WARRANTY; without even the implied warranty of
all: update license headers to distiguish GPL/LGPL All code outside of cmd/ is licensed as LGPL. The headers now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 11:48:40 -05:00
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
all: update license information
2015-07-06 19:54:22 -05:00
// GNU Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
all: update license headers to distiguish GPL/LGPL All code outside of cmd/ is licensed as LGPL. The headers now reflect this by calling the whole work "the go-ethereum library".
2015-07-22 11:48:40 -05:00
// along with the go-ethereum library. If not, see <http://www.gnu.org/licenses/>.
all: update license information
2015-07-06 19:54:22 -05:00
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
package rlp
import (
"bytes"
"errors"
"fmt"
"io"
"math/big"
rlp: use atomic.Value for type cache (#22902) All encoding/decoding operations read the type cache to find the writer/decoder function responsible for a type. When analyzing CPU profiles of geth during sync, I found that the use of sync.RWMutex in cache lookups appears in the profiles. It seems we are running into CPU cache contention problems when package rlp is heavily used on all CPU cores during sync. This change makes it use atomic.Value + a writer lock instead of sync.RWMutex. In the common case where the typeinfo entry is present in the cache, we simply fetch the map and lookup the type.
2021-05-22 06:34:29 -05:00
"runtime"
rlp: fix encReader returning nil buffers to the pool The bug can cause crashes if Read is called after EOF has been returned. No code performs such calls right now, but hitting the bug gets more likely as rlp.EncodeToReader gets used in more places.
2015-09-02 18:37:00 -05:00
"sync"
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
"testing"
rlp: reduce allocations for big.Int and byte array encoding (#21291) This change further improves the performance of RLP encoding by removing allocations for big.Int and [...]byte types. I have added a new benchmark that measures RLP encoding of types.Block to verify that performance is improved.
2020-07-06 04:17:09 -05:00
"github.com/ethereum/go-ethereum/common/math"
rlp: support for uint256 (#26898) This adds built-in support in package rlp for encoding, decoding and generating code dealing with uint256.Int. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-17 05:51:55 -05:00
"github.com/holiman/uint256"
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
)
type testEncoder struct {
err error
}
func (e *testEncoder) EncodeRLP(w io.Writer) error {
if e == nil {
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
panic("EncodeRLP called on nil value")
}
if e.err != nil {
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
return e.err
}
all: simplify nested complexity and if blocks ending with a return statement (#21854) Changes: Simplify nested complexity If an if blocks ends with a return statement then remove the else nesting. Most of the changes has also been reported in golint https://goreportcard.com/report/github.com/ethereum/go-ethereum#golint
2020-11-25 02:24:50 -06:00
w.Write([]byte{0, 1, 0, 1, 0, 1, 0, 1, 0, 1})
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
return nil
}
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
type testEncoderValueMethod struct{}
func (e testEncoderValueMethod) EncodeRLP(w io.Writer) error {
w.Write([]byte{0xFA, 0xFE, 0xF0})
return nil
}
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
type byteEncoder byte
func (e byteEncoder) EncodeRLP(w io.Writer) error {
w.Write(EmptyList)
return nil
}
rlp: fixes for two corner cases and documentation (#19527) These changes fix two corner cases related to internal handling of types in package rlp: The "tail" struct tag can only be applied to the last field. The check for this was wrong and didn't allow for private fields after the field with the tag. Unsupported types (e.g. structs containing int) which implement either the Encoder or Decoder interface but not both couldn't be encoded/decoded. Also fixes #19367
2019-05-14 08:09:56 -05:00
type undecodableEncoder func()
func (f undecodableEncoder) EncodeRLP(w io.Writer) error {
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
w.Write([]byte{0xF5, 0xF5, 0xF5})
return nil
rlp: fixes for two corner cases and documentation (#19527) These changes fix two corner cases related to internal handling of types in package rlp: The "tail" struct tag can only be applied to the last field. The check for this was wrong and didn't allow for private fields after the field with the tag. Unsupported types (e.g. structs containing int) which implement either the Encoder or Decoder interface but not both couldn't be encoded/decoded. Also fixes #19367
2019-05-14 08:09:56 -05:00
}
rlp: allow encoding non-empty interface values This needs to be supported because []someInterface does occur sometimes. Funny enough, the fix involves changes to the decoder. makeDecoder cannot return an error for non-empty interfaces anymore because the type cache builds both decoder and writer. Do the check at 'runtime' instead.
2015-01-15 16:21:41 -06:00
type encodableReader struct {
A, B uint
}
func (e *encodableReader) Read(b []byte) (int, error) {
panic("called")
}
rlp: fix encoding of arrays with byte element type
2015-01-30 09:52:48 -06:00
type namedByteType byte
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
var (
_ = Encoder(&testEncoder{})
_ = Encoder(byteEncoder(0))
rlp: allow encoding non-empty interface values This needs to be supported because []someInterface does occur sometimes. Funny enough, the fix involves changes to the decoder. makeDecoder cannot return an error for non-empty interfaces anymore because the type cache builds both decoder and writer. Do the check at 'runtime' instead.
2015-01-15 16:21:41 -06:00
reader io.Reader = &encodableReader{1, 2}
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
)
type encTest struct {
val interface{}
output, error string
}
var encTests = []encTest{
rlp: add support for boolean encoding/decoding
2015-08-13 03:12:38 -05:00
// booleans
{val: true, output: "01"},
{val: false, output: "80"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
// integers
{val: uint32(0), output: "80"},
{val: uint32(127), output: "7F"},
{val: uint32(128), output: "8180"},
{val: uint32(256), output: "820100"},
{val: uint32(1024), output: "820400"},
{val: uint32(0xFFFFFF), output: "83FFFFFF"},
{val: uint32(0xFFFFFFFF), output: "84FFFFFFFF"},
{val: uint64(0xFFFFFFFF), output: "84FFFFFFFF"},
{val: uint64(0xFFFFFFFFFF), output: "85FFFFFFFFFF"},
{val: uint64(0xFFFFFFFFFFFF), output: "86FFFFFFFFFFFF"},
{val: uint64(0xFFFFFFFFFFFFFF), output: "87FFFFFFFFFFFFFF"},
{val: uint64(0xFFFFFFFFFFFFFFFF), output: "88FFFFFFFFFFFFFFFF"},
// big integers (should match uint for small values)
{val: big.NewInt(0), output: "80"},
{val: big.NewInt(1), output: "01"},
{val: big.NewInt(127), output: "7F"},
{val: big.NewInt(128), output: "8180"},
{val: big.NewInt(256), output: "820100"},
{val: big.NewInt(1024), output: "820400"},
{val: big.NewInt(0xFFFFFF), output: "83FFFFFF"},
{val: big.NewInt(0xFFFFFFFF), output: "84FFFFFFFF"},
{val: big.NewInt(0xFFFFFFFFFF), output: "85FFFFFFFFFF"},
{val: big.NewInt(0xFFFFFFFFFFFF), output: "86FFFFFFFFFFFF"},
{val: big.NewInt(0xFFFFFFFFFFFFFF), output: "87FFFFFFFFFFFFFF"},
{
all: prefer `new(big.Int)` over `big.NewInt(0)` (#25087) minor performance improvement: `big.NewInt(0).Xxx` -> `new(big.Int).Xxx`
2022-06-14 07:09:48 -05:00
val: new(big.Int).SetBytes(unhex("102030405060708090A0B0C0D0E0F2")),
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
output: "8F102030405060708090A0B0C0D0E0F2",
},
{
all: prefer `new(big.Int)` over `big.NewInt(0)` (#25087) minor performance improvement: `big.NewInt(0).Xxx` -> `new(big.Int).Xxx`
2022-06-14 07:09:48 -05:00
val: new(big.Int).SetBytes(unhex("0100020003000400050006000700080009000A000B000C000D000E01")),
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
output: "9C0100020003000400050006000700080009000A000B000C000D000E01",
},
{
all: prefer `new(big.Int)` over `big.NewInt(0)` (#25087) minor performance improvement: `big.NewInt(0).Xxx` -> `new(big.Int).Xxx`
2022-06-14 07:09:48 -05:00
val: new(big.Int).SetBytes(unhex("010000000000000000000000000000000000000000000000000000000000000000")),
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
output: "A1010000000000000000000000000000000000000000000000000000000000000000",
},
rlp: optimize big.Int decoding for size <= 32 bytes (#22927) This change grows the static integer buffer in Stream to 32 bytes, making it possible to decode 256bit integers without allocating a temporary buffer. In the recent commit 088da24, Stream struct size decreased from 120 bytes down to 88 bytes. This commit grows the struct to 112 bytes again, but the size change will not degrade performance because Stream instances are internally cached in sync.Pool. name old time/op new time/op delta DecodeBigInts-8 12.2µs ± 0% 8.6µs ± 4% -29.58% (p=0.000 n=9+10) name old speed new speed delta DecodeBigInts-8 230MB/s ± 0% 326MB/s ± 4% +42.04% (p=0.000 n=9+10)
2021-05-25 14:56:25 -05:00
{
val: veryBigInt,
output: "89FFFFFFFFFFFFFFFFFF",
},
{
val: veryVeryBigInt,
output: "B848FFFFFFFFFFFFFFFFF800000000000000001BFFFFFFFFFFFFFFFFC8000000000000000045FFFFFFFFFFFFFFFFC800000000000000001BFFFFFFFFFFFFFFFFF8000000000000000001",
},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
// non-pointer big.Int
{val: *big.NewInt(0), output: "80"},
{val: *big.NewInt(0xFFFFFF), output: "83FFFFFF"},
// negative ints are not supported
rlp/rlpgen: RLP encoder code generator (#24251) This change adds a code generator tool for creating EncodeRLP method implementations. The generated methods will behave identically to the reflect-based encoder, but run faster because there is no reflection overhead. Package rlp now provides the EncoderBuffer type for incremental encoding. This is used by generated code, but the new methods can also be useful for hand-written encoders. There is also experimental support for generating DecodeRLP, and some new methods have been added to the existing Stream type to support this. Creating decoders with rlpgen is not recommended at this time because the generated methods create very poor error reporting. More detail about package rlp changes: * rlp: externalize struct field processing / validation This adds a new package, rlp/internal/rlpstruct, in preparation for the RLP encoder generator. I think the struct field rules are subtle enough to warrant extracting this into their own package, even though it means that a bunch of adapter code is needed for converting to/from rlpstruct.Type. * rlp: add more decoder methods (for rlpgen) This adds new methods on rlp.Stream: - Uint64, Uint32, Uint16, Uint8, BigInt - ReadBytes for decoding into []byte - MoreDataInList - useful for optional list elements * rlp: expose encoder buffer (for rlpgen) This exposes the internal encoder buffer type for use in EncodeRLP implementations. The new EncoderBuffer type is a sort-of 'opaque handle' for a pointer to encBuffer. It is implemented this way to ensure the global encBuffer pool is handled correctly.
2022-02-16 11:14:12 -06:00
{val: big.NewInt(-1), error: "rlp: cannot encode negative big.Int"},
{val: *big.NewInt(-1), error: "rlp: cannot encode negative big.Int"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
rlp: support for uint256 (#26898) This adds built-in support in package rlp for encoding, decoding and generating code dealing with uint256.Int. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-17 05:51:55 -05:00
// uint256
{val: uint256.NewInt(0), output: "80"},
{val: uint256.NewInt(1), output: "01"},
{val: uint256.NewInt(127), output: "7F"},
{val: uint256.NewInt(128), output: "8180"},
{val: uint256.NewInt(256), output: "820100"},
{val: uint256.NewInt(1024), output: "820400"},
{val: uint256.NewInt(0xFFFFFF), output: "83FFFFFF"},
{val: uint256.NewInt(0xFFFFFFFF), output: "84FFFFFFFF"},
{val: uint256.NewInt(0xFFFFFFFFFF), output: "85FFFFFFFFFF"},
{val: uint256.NewInt(0xFFFFFFFFFFFF), output: "86FFFFFFFFFFFF"},
{val: uint256.NewInt(0xFFFFFFFFFFFFFF), output: "87FFFFFFFFFFFFFF"},
{
val: new(uint256.Int).SetBytes(unhex("102030405060708090A0B0C0D0E0F2")),
output: "8F102030405060708090A0B0C0D0E0F2",
},
{
val: new(uint256.Int).SetBytes(unhex("0100020003000400050006000700080009000A000B000C000D000E01")),
output: "9C0100020003000400050006000700080009000A000B000C000D000E01",
},
// non-pointer uint256.Int
{val: *uint256.NewInt(0), output: "80"},
{val: *uint256.NewInt(0xFFFFFF), output: "83FFFFFF"},
rlp: reduce allocations for big.Int and byte array encoding (#21291) This change further improves the performance of RLP encoding by removing allocations for big.Int and [...]byte types. I have added a new benchmark that measures RLP encoding of types.Block to verify that performance is improved.
2020-07-06 04:17:09 -05:00
// byte arrays
{val: [0]byte{}, output: "80"},
{val: [1]byte{0}, output: "00"},
{val: [1]byte{1}, output: "01"},
{val: [1]byte{0x7F}, output: "7F"},
{val: [1]byte{0x80}, output: "8180"},
{val: [1]byte{0xFF}, output: "81FF"},
{val: [3]byte{1, 2, 3}, output: "83010203"},
{val: [57]byte{1, 2, 3}, output: "B839010203000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"},
// named byte type arrays
{val: [0]namedByteType{}, output: "80"},
{val: [1]namedByteType{0}, output: "00"},
{val: [1]namedByteType{1}, output: "01"},
{val: [1]namedByteType{0x7F}, output: "7F"},
{val: [1]namedByteType{0x80}, output: "8180"},
{val: [1]namedByteType{0xFF}, output: "81FF"},
{val: [3]namedByteType{1, 2, 3}, output: "83010203"},
{val: [57]namedByteType{1, 2, 3}, output: "B839010203000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"},
// byte slices
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: []byte{}, output: "80"},
rlp: reduce allocations for big.Int and byte array encoding (#21291) This change further improves the performance of RLP encoding by removing allocations for big.Int and [...]byte types. I have added a new benchmark that measures RLP encoding of types.Block to verify that performance is improved.
2020-07-06 04:17:09 -05:00
{val: []byte{0}, output: "00"},
rlp: fix encoding of one element strings and byte slices The encoder was missing a special case for one element strings whose element is below 0x7f. Such strings must be encoded as a single byte without a string header.
2015-03-19 06:15:43 -05:00
{val: []byte{0x7E}, output: "7E"},
{val: []byte{0x7F}, output: "7F"},
{val: []byte{0x80}, output: "8180"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: []byte{1, 2, 3}, output: "83010203"},
rlp: fix encoding of arrays with byte element type
2015-01-30 09:52:48 -06:00
rlp: reduce allocations for big.Int and byte array encoding (#21291) This change further improves the performance of RLP encoding by removing allocations for big.Int and [...]byte types. I have added a new benchmark that measures RLP encoding of types.Block to verify that performance is improved.
2020-07-06 04:17:09 -05:00
// named byte type slices
{val: []namedByteType{}, output: "80"},
{val: []namedByteType{0}, output: "00"},
{val: []namedByteType{0x7E}, output: "7E"},
{val: []namedByteType{0x7F}, output: "7F"},
{val: []namedByteType{0x80}, output: "8180"},
rlp: fix encoding of arrays with byte element type
2015-01-30 09:52:48 -06:00
{val: []namedByteType{1, 2, 3}, output: "83010203"},
rlp: reduce allocations for big.Int and byte array encoding (#21291) This change further improves the performance of RLP encoding by removing allocations for big.Int and [...]byte types. I have added a new benchmark that measures RLP encoding of types.Block to verify that performance is improved.
2020-07-06 04:17:09 -05:00
// strings
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: "", output: "80"},
rlp: fix encoding of one element strings and byte slices The encoder was missing a special case for one element strings whose element is below 0x7f. Such strings must be encoded as a single byte without a string header.
2015-03-19 06:15:43 -05:00
{val: "\x7E", output: "7E"},
{val: "\x7F", output: "7F"},
{val: "\x80", output: "8180"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: "dog", output: "83646F67"},
{
val: "Lorem ipsum dolor sit amet, consectetur adipisicing eli",
output: "B74C6F72656D20697073756D20646F6C6F722073697420616D65742C20636F6E7365637465747572206164697069736963696E6720656C69",
},
{
val: "Lorem ipsum dolor sit amet, consectetur adipisicing elit",
output: "B8384C6F72656D20697073756D20646F6C6F722073697420616D65742C20636F6E7365637465747572206164697069736963696E6720656C6974",
},
{
val: "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur mauris magna, suscipit sed vehicula non, iaculis faucibus tortor. Proin suscipit ultricies malesuada. Duis tortor elit, dictum quis tristique eu, ultrices at risus. Morbi a est imperdiet mi ullamcorper aliquet suscipit nec lorem. Aenean quis leo mollis, vulputate elit varius, consequat enim. Nulla ultrices turpis justo, et posuere urna consectetur nec. Proin non convallis metus. Donec tempor ipsum in mauris congue sollicitudin. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Suspendisse convallis sem vel massa faucibus, eget lacinia lacus tempor. Nulla quis ultricies purus. Proin auctor rhoncus nibh condimentum mollis. Aliquam consequat enim at metus luctus, a eleifend purus egestas. Curabitur at nibh metus. Nam bibendum, neque at auctor tristique, lorem libero aliquet arcu, non interdum tellus lectus sit amet eros. Cras rhoncus, metus ac ornare cursus, dolor justo ultrices metus, at ullamcorper volutpat",
output: "B904004C6F72656D20697073756D20646F6C6F722073697420616D65742C20636F6E73656374657475722061646970697363696E6720656C69742E20437572616269747572206D6175726973206D61676E612C20737573636970697420736564207665686963756C61206E6F6E2C20696163756C697320666175636962757320746F72746F722E2050726F696E20737573636970697420756C74726963696573206D616C6573756164612E204475697320746F72746F7220656C69742C2064696374756D2071756973207472697374697175652065752C20756C7472696365732061742072697375732E204D6F72626920612065737420696D70657264696574206D6920756C6C616D636F7270657220616C6971756574207375736369706974206E6563206C6F72656D2E2041656E65616E2071756973206C656F206D6F6C6C69732C2076756C70757461746520656C6974207661726975732C20636F6E73657175617420656E696D2E204E756C6C6120756C74726963657320747572706973206A7573746F2C20657420706F73756572652075726E6120636F6E7365637465747572206E65632E2050726F696E206E6F6E20636F6E76616C6C6973206D657475732E20446F6E65632074656D706F7220697073756D20696E206D617572697320636F6E67756520736F6C6C696369747564696E2E20566573746962756C756D20616E746520697073756D207072696D697320696E206661756369627573206F726369206C756374757320657420756C74726963657320706F737565726520637562696C69612043757261653B2053757370656E646973736520636F6E76616C6C69732073656D2076656C206D617373612066617563696275732C2065676574206C6163696E6961206C616375732074656D706F722E204E756C6C61207175697320756C747269636965732070757275732E2050726F696E20617563746F722072686F6E637573206E69626820636F6E64696D656E74756D206D6F6C6C69732E20416C697175616D20636F6E73657175617420656E696D206174206D65747573206C75637475732C206120656C656966656E6420707572757320656765737461732E20437572616269747572206174206E696268206D657475732E204E616D20626962656E64756D2C206E6571756520617420617563746F72207472697374697175652C206C6F72656D206C696265726F20616C697175657420617263752C206E6F6E20696E74657264756D2074656C6C7573206C65637475732073697420616D65742065726F732E20437261732072686F6E6375732C206D65747573206163206F726E617265206375727375732C20646F6C6F72206A7573746F20756C747269636573206D657475732C20617420756C6C616D636F7270657220766F6C7574706174",
},
// slices
{val: []uint{}, output: "C0"},
{val: []uint{1, 2, 3}, output: "C3010203"},
{
// [ [], [[]], [ [], [[]] ] ]
val: []interface{}{[]interface{}{}, [][]interface{}{{}}, []interface{}{[]interface{}{}, [][]interface{}{{}}}},
output: "C7C0C1C0C3C0C1C0",
},
{
val: []string{"aaa", "bbb", "ccc", "ddd", "eee", "fff", "ggg", "hhh", "iii", "jjj", "kkk", "lll", "mmm", "nnn", "ooo"},
output: "F83C836161618362626283636363836464648365656583666666836767678368686883696969836A6A6A836B6B6B836C6C6C836D6D6D836E6E6E836F6F6F",
},
{
val: []interface{}{uint(1), uint(0xFFFFFF), []interface{}{[]uint{4, 5, 5}}, "abc"},
output: "CE0183FFFFFFC4C304050583616263",
},
{
val: [][]string{
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
{"asdf", "qwer", "zxcv"},
},
output: "F90200CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376CF84617364668471776572847A786376",
},
rlp: support for uint256 (#26898) This adds built-in support in package rlp for encoding, decoding and generating code dealing with uint256.Int. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-17 05:51:55 -05:00
// Non-byte arrays are encoded as lists.
// Note that it is important to test [4]uint64 specifically,
// because that's the underlying type of uint256.Int.
{val: [4]uint32{1, 2, 3, 4}, output: "C401020304"},
{val: [4]uint64{1, 2, 3, 4}, output: "C401020304"},
rlp: add RawValue
2015-09-08 20:34:15 -05:00
// RawValue
{val: RawValue(unhex("01")), output: "01"},
{val: RawValue(unhex("82FFFF")), output: "82FFFF"},
{val: []RawValue{unhex("01"), unhex("02")}, output: "C20102"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
// structs
{val: simplestruct{}, output: "C28080"},
{val: simplestruct{A: 3, B: "foo"}, output: "C50383666F6F"},
{val: &recstruct{5, nil}, output: "C205C0"},
{val: &recstruct{5, &recstruct{4, &recstruct{3, nil}}}, output: "C605C404C203C0"},
rlp: add support for optional struct fields (#22832) This adds support for a new struct tag "optional". Using this tag, structs used for RLP encoding/decoding can be extended in a backwards-compatible way, by adding new fields at the end.
2021-05-07 07:37:13 -05:00
{val: &intField{X: 3}, error: "rlp: type int is not RLP-serializable (struct field rlp.intField.X)"},
// struct tag "-"
{val: &ignoredField{A: 1, B: 2, C: 3}, output: "C20103"},
// struct tag "tail"
rlp: add "tail" struct tag
2015-12-21 14:05:20 -06:00
{val: &tailRaw{A: 1, Tail: []RawValue{unhex("02"), unhex("03")}}, output: "C3010203"},
{val: &tailRaw{A: 1, Tail: []RawValue{unhex("02")}}, output: "C20102"},
{val: &tailRaw{A: 1, Tail: []RawValue{}}, output: "C101"},
{val: &tailRaw{A: 1, Tail: nil}, output: "C101"},
rlp: add support for optional struct fields (#22832) This adds support for a new struct tag "optional". Using this tag, structs used for RLP encoding/decoding can be extended in a backwards-compatible way, by adding new fields at the end.
2021-05-07 07:37:13 -05:00
// struct tag "optional"
{val: &optionalFields{}, output: "C180"},
{val: &optionalFields{A: 1}, output: "C101"},
{val: &optionalFields{A: 1, B: 2}, output: "C20102"},
{val: &optionalFields{A: 1, B: 2, C: 3}, output: "C3010203"},
{val: &optionalFields{A: 1, B: 0, C: 3}, output: "C3018003"},
{val: &optionalAndTailField{A: 1}, output: "C101"},
{val: &optionalAndTailField{A: 1, B: 2}, output: "C20102"},
{val: &optionalAndTailField{A: 1, Tail: []uint{5, 6}}, output: "C401800506"},
{val: &optionalAndTailField{A: 1, Tail: []uint{5, 6}}, output: "C401800506"},
{val: &optionalBigIntField{A: 1}, output: "C101"},
{val: &optionalPtrField{A: 1}, output: "C101"},
{val: &optionalPtrFieldNil{A: 1}, output: "C101"},
rlp: add more tests for nil pointer / optional field encoding (#26077)
2022-11-02 08:50:48 -05:00
{val: &multipleOptionalFields{A: nil, B: nil}, output: "C0"},
{val: &multipleOptionalFields{A: &[3]byte{1, 2, 3}, B: &[3]byte{1, 2, 3}}, output: "C88301020383010203"},
{val: &multipleOptionalFields{A: nil, B: &[3]byte{1, 2, 3}}, output: "C58083010203"}, // encodes without error but decode will fail
{val: &nonOptionalPtrField{A: 1}, output: "C20180"}, // encodes without error but decode will fail
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
// nil
{val: (*uint)(nil), output: "80"},
{val: (*string)(nil), output: "80"},
{val: (*[]byte)(nil), output: "80"},
rlp: encode nil array pointers as empty list or string
2015-03-25 10:46:29 -05:00
{val: (*[10]byte)(nil), output: "80"},
rlp: don't panic for nil *big.Int All other pointer types can handle nil just fine.
2015-03-17 17:49:49 -05:00
{val: (*big.Int)(nil), output: "80"},
rlp: support for uint256 (#26898) This adds built-in support in package rlp for encoding, decoding and generating code dealing with uint256.Int. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-17 05:51:55 -05:00
{val: (*uint256.Int)(nil), output: "80"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: (*[]string)(nil), output: "C0"},
rlp: encode nil array pointers as empty list or string
2015-03-25 10:46:29 -05:00
{val: (*[10]string)(nil), output: "C0"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: (*[]interface{})(nil), output: "C0"},
{val: (*[]struct{ uint })(nil), output: "C0"},
{val: (*interface{})(nil), output: "C0"},
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
// nil struct fields
{
val: struct {
X *[]byte
}{},
output: "C180",
},
{
val: struct {
X *[2]byte
}{},
output: "C180",
},
{
val: struct {
X *uint64
}{},
output: "C180",
},
{
val: struct {
X *uint64 `rlp:"nilList"`
}{},
output: "C1C0",
},
{
val: struct {
X *[]uint64
}{},
output: "C1C0",
},
{
val: struct {
X *[]uint64 `rlp:"nilString"`
}{},
output: "C180",
},
rlp: allow encoding non-empty interface values This needs to be supported because []someInterface does occur sometimes. Funny enough, the fix involves changes to the decoder. makeDecoder cannot return an error for non-empty interfaces anymore because the type cache builds both decoder and writer. Do the check at 'runtime' instead.
2015-01-15 16:21:41 -06:00
// interfaces
{val: []io.Reader{reader}, output: "C3C20102"}, // the contained value is a struct
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
// Encoder
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
{val: (*testEncoder)(nil), output: "C0"},
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: &testEncoder{}, output: "00010001000100010001"},
{val: &testEncoder{errors.New("test error")}, error: "test error"},
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
{val: struct{ E testEncoderValueMethod }{}, output: "C3FAFEF0"},
{val: struct{ E *testEncoderValueMethod }{}, output: "C1C0"},
// Verify that the Encoder interface works for unsupported types like func().
{val: undecodableEncoder(func() {}), output: "F5F5F5"},
// Verify that pointer method testEncoder.EncodeRLP is called for
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
// addressable non-pointer values.
{val: &struct{ TE testEncoder }{testEncoder{}}, output: "CA00010001000100010001"},
{val: &struct{ TE testEncoder }{testEncoder{errors.New("test error")}}, error: "test error"},
rlp: improve nil pointer handling (#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test.
2019-09-13 04:10:57 -05:00
// Verify the error for non-addressable non-pointer Encoder.
{val: testEncoder{}, error: "rlp: unadressable value of type rlp.testEncoder, EncodeRLP is pointer method"},
// Verify Encoder takes precedence over []byte.
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
{val: []byteEncoder{0, 1, 2, 3, 4}, output: "C5C0C0C0C0C0"},
}
func runEncTests(t *testing.T, f func(val interface{}) ([]byte, error)) {
for i, test := range encTests {
output, err := f(test.val)
if err != nil && test.error == "" {
t.Errorf("test %d: unexpected error: %v\nvalue %#v\ntype %T",
i, err, test.val, test.val)
continue
}
if test.error != "" && fmt.Sprint(err) != test.error {
t.Errorf("test %d: error mismatch\ngot %v\nwant %v\nvalue %#v\ntype %T",
i, err, test.error, test.val, test.val)
continue
}
if err == nil && !bytes.Equal(output, unhex(test.output)) {
t.Errorf("test %d: output mismatch:\ngot %X\nwant %s\nvalue %#v\ntype %T",
i, output, test.output, test.val, test.val)
}
}
}
func TestEncode(t *testing.T) {
runEncTests(t, func(val interface{}) ([]byte, error) {
b := new(bytes.Buffer)
err := Encode(b, val)
return b.Bytes(), err
})
}
func TestEncodeToBytes(t *testing.T) {
runEncTests(t, EncodeToBytes)
}
rlp, trie: faster trie node encoding (#24126) This change speeds up trie hashing and all other activities that require RLP encoding of trie nodes by approximately 20%. The speedup is achieved by avoiding reflection overhead during node encoding. The interface type trie.node now contains a method 'encode' that works with rlp.EncoderBuffer. Management of EncoderBuffers is left to calling code. trie.hasher, which is pooled to avoid allocations, now maintains an EncoderBuffer. This means memory resources related to trie node encoding are tied to the hasher pool. Co-authored-by: Felix Lange <fjl@twurst.com>
2022-03-09 07:45:17 -06:00
func TestEncodeAppendToBytes(t *testing.T) {
buffer := make([]byte, 20)
runEncTests(t, func(val interface{}) ([]byte, error) {
w := NewEncoderBuffer(nil)
defer w.Flush()
err := Encode(w, val)
if err != nil {
return nil, err
}
output := w.AppendToBytes(buffer[:0])
return output, nil
})
}
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
func TestEncodeToReader(t *testing.T) {
runEncTests(t, func(val interface{}) ([]byte, error) {
_, r, err := EncodeToReader(val)
if err != nil {
return nil, err
}
all: replace uses of ioutil with io and os (#24869)
2022-05-16 04:59:35 -05:00
return io.ReadAll(r)
rlp: add functions for encoding I'm reasonably confident that the encoding matches the output of ethutil.Encode for values that it supports. Some of the tests have been adpated from the Ethereum testing repository. There are still TODOs in the code.
2014-12-30 04:40:32 -06:00
})
}
func TestEncodeToReaderPiecewise(t *testing.T) {
runEncTests(t, func(val interface{}) ([]byte, error) {
size, r, err := EncodeToReader(val)
if err != nil {
return nil, err
}
// read output piecewise
output := make([]byte, size)
for start, end := 0, 0; start < size; start = end {
if remaining := size - start; remaining < 3 {
end += remaining
} else {
end = start + 3
}
n, err := r.Read(output[start:end])
end = start + n
if err == io.EOF {
break
} else if err != nil {
return nil, err
}
}
return output, nil
})
}
rlp: fix encReader returning nil buffers to the pool The bug can cause crashes if Read is called after EOF has been returned. No code performs such calls right now, but hitting the bug gets more likely as rlp.EncodeToReader gets used in more places.
2015-09-02 18:37:00 -05:00
// This is a regression test verifying that encReader
// returns its encbuf to the pool only once.
func TestEncodeToReaderReturnToPool(t *testing.T) {
buf := make([]byte, 50)
wg := new(sync.WaitGroup)
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
for i := 0; i < 1000; i++ {
_, r, _ := EncodeToReader("foo")
all: replace uses of ioutil with io and os (#24869)
2022-05-16 04:59:35 -05:00
io.ReadAll(r)
rlp: fix encReader returning nil buffers to the pool The bug can cause crashes if Read is called after EOF has been returned. No code performs such calls right now, but hitting the bug gets more likely as rlp.EncodeToReader gets used in more places.
2015-09-02 18:37:00 -05:00
r.Read(buf)
r.Read(buf)
r.Read(buf)
r.Read(buf)
}
wg.Done()
}()
}
wg.Wait()
}
rlp: reduce allocations for big.Int and byte array encoding (#21291) This change further improves the performance of RLP encoding by removing allocations for big.Int and [...]byte types. I have added a new benchmark that measures RLP encoding of types.Block to verify that performance is improved.
2020-07-06 04:17:09 -05:00
var sink interface{}
func BenchmarkIntsize(b *testing.B) {
for i := 0; i < b.N; i++ {
sink = intsize(0x12345678)
}
}
func BenchmarkPutint(b *testing.B) {
buf := make([]byte, 8)
for i := 0; i < b.N; i++ {
putint(buf, 0x12345678)
sink = buf
}
}
func BenchmarkEncodeBigInts(b *testing.B) {
ints := make([]*big.Int, 200)
for i := range ints {
ints[i] = math.BigPow(2, int64(i))
}
out := bytes.NewBuffer(make([]byte, 0, 4096))
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
out.Reset()
if err := Encode(out, ints); err != nil {
b.Fatal(err)
}
}
}
rlp: use atomic.Value for type cache (#22902) All encoding/decoding operations read the type cache to find the writer/decoder function responsible for a type. When analyzing CPU profiles of geth during sync, I found that the use of sync.RWMutex in cache lookups appears in the profiles. It seems we are running into CPU cache contention problems when package rlp is heavily used on all CPU cores during sync. This change makes it use atomic.Value + a writer lock instead of sync.RWMutex. In the common case where the typeinfo entry is present in the cache, we simply fetch the map and lookup the type.
2021-05-22 06:34:29 -05:00
rlp: support for uint256 (#26898) This adds built-in support in package rlp for encoding, decoding and generating code dealing with uint256.Int. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
2023-03-17 05:51:55 -05:00
func BenchmarkEncodeU256Ints(b *testing.B) {
ints := make([]*uint256.Int, 200)
for i := range ints {
ints[i], _ = uint256.FromBig(math.BigPow(2, int64(i)))
}
out := bytes.NewBuffer(make([]byte, 0, 4096))
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
out.Reset()
if err := Encode(out, ints); err != nil {
b.Fatal(err)
}
}
}
rlp: use atomic.Value for type cache (#22902) All encoding/decoding operations read the type cache to find the writer/decoder function responsible for a type. When analyzing CPU profiles of geth during sync, I found that the use of sync.RWMutex in cache lookups appears in the profiles. It seems we are running into CPU cache contention problems when package rlp is heavily used on all CPU cores during sync. This change makes it use atomic.Value + a writer lock instead of sync.RWMutex. In the common case where the typeinfo entry is present in the cache, we simply fetch the map and lookup the type.
2021-05-22 06:34:29 -05:00
func BenchmarkEncodeConcurrentInterface(b *testing.B) {
type struct1 struct {
A string
B *big.Int
C [20]byte
}
value := []interface{}{
uint(999),
&struct1{A: "hello", B: big.NewInt(0xFFFFFFFF)},
[10]byte{1, 2, 3, 4, 5, 6},
[]string{"yeah", "yeah", "yeah"},
}
var wg sync.WaitGroup
for cpu := 0; cpu < runtime.NumCPU(); cpu++ {
wg.Add(1)
go func() {
defer wg.Done()
var buffer bytes.Buffer
for i := 0; i < b.N; i++ {
buffer.Reset()
err := Encode(&buffer, value)
if err != nil {
panic(err)
}
}
}()
}
wg.Wait()
}
rlp: optimize byte array handling (#22924) This change improves the performance of encoding/decoding [N]byte. name old time/op new time/op delta DecodeByteArrayStruct-8 336ns ± 0% 246ns ± 0% -26.98% (p=0.000 n=9+10) EncodeByteArrayStruct-8 225ns ± 1% 148ns ± 1% -34.12% (p=0.000 n=10+10) name old alloc/op new alloc/op delta DecodeByteArrayStruct-8 120B ± 0% 48B ± 0% -60.00% (p=0.000 n=10+10) EncodeByteArrayStruct-8 0.00B 0.00B ~ (all equal)
2021-05-22 08:10:16 -05:00
type byteArrayStruct struct {
A [20]byte
B [32]byte
C [32]byte
}
func BenchmarkEncodeByteArrayStruct(b *testing.B) {
var out bytes.Buffer
var value byteArrayStruct
b.ReportAllocs()
for i := 0; i < b.N; i++ {
out.Reset()
if err := Encode(&out, &value); err != nil {
b.Fatal(err)
}
}
}
rlp: minor optimizations for slice/array encoding (#23467) As per benchmark results below, these changes speed up encoding/decoding of consensus objects a bit. name old time/op new time/op delta EncodeRLP/legacy-header-8 384ns ± 1% 331ns ± 3% -13.83% (p=0.000 n=7+8) EncodeRLP/london-header-8 411ns ± 1% 359ns ± 2% -12.53% (p=0.000 n=8+8) EncodeRLP/receipt-for-storage-8 251ns ± 0% 239ns ± 0% -4.97% (p=0.000 n=8+8) EncodeRLP/receipt-full-8 319ns ± 0% 300ns ± 0% -5.89% (p=0.000 n=8+7) EncodeRLP/legacy-transaction-8 389ns ± 1% 387ns ± 1% ~ (p=0.099 n=8+8) EncodeRLP/access-transaction-8 607ns ± 0% 581ns ± 0% -4.26% (p=0.000 n=8+8) EncodeRLP/1559-transaction-8 627ns ± 0% 606ns ± 1% -3.44% (p=0.000 n=8+8) DecodeRLP/legacy-header-8 831ns ± 1% 813ns ± 1% -2.20% (p=0.000 n=8+8) DecodeRLP/london-header-8 824ns ± 0% 804ns ± 1% -2.44% (p=0.000 n=8+7) * rlp: pass length to byteArrayBytes This makes it possible to inline byteArrayBytes. For arrays, the length is known at encoder construction time, so the call to v.Len() can be avoided. * rlp: avoid IsNil for pointer encoding It's actually cheaper to use Elem first, because it performs less checks on the value. If the pointer was nil, the result of Elem is 'invalid'. * rlp: minor optimizations for slice/array encoding For empty slices/arrays, we can avoid storing a list header entry in the encoder buffer. Also avoid doing the tail check at encoding time because it is already known at encoder construction time.
2021-08-25 12:01:10 -05:00
type structSliceElem struct {
X uint64
Y uint64
Z uint64
}
type structPtrSlice []*structSliceElem
func BenchmarkEncodeStructPtrSlice(b *testing.B) {
var out bytes.Buffer
var value = structPtrSlice{
&structSliceElem{1, 1, 1},
&structSliceElem{2, 2, 2},
&structSliceElem{3, 3, 3},
&structSliceElem{5, 5, 5},
&structSliceElem{6, 6, 6},
&structSliceElem{7, 7, 7},
}
b.ReportAllocs()
for i := 0; i < b.N; i++ {
out.Reset()
if err := Encode(&out, &value); err != nil {
b.Fatal(err)
}
}
}