garbage collector found invalid heap pointer: Go 1.4 #163

New Issue

2015-01-01T10:50:20-06:00

HawaiianSpork commented

2015-01-01 10:50:20 -06:00

(Migrated from github.com)

After upgrading from go1.3 to go1.4, I'd get a runtime crash most of the time. It crashes for various reasons (nil tree walker callback, invalid heap pointer...). Here is a scaled down version of the code that demonstrates the problem (runtime crashes most of the time). Run this code with an argument pointing at a git repository like git2go.

package main

import (
    "fmt"
    "github.com/libgit2/git2go"
    "os"
    "strings"
)

type MyRepo struct {
    *git.Repository
}

func (repository *MyRepo) foo(path string, treeEntry *git.TreeEntry) int {
    if git.ObjectBlob != treeEntry.Type {
        return 0 // continue
    }

    if !strings.HasSuffix(treeEntry.Name, ".go") {
        return 0 // continue
    }

    blob, err := repository.Repository.LookupBlob(treeEntry.Id)
    defer blob.Free()
    if err != nil {
        panic(err)
    }

    if blob.Size() > 10000 {
        return 0 // continue
    }

    contents := blob.Contents()
    if contents == nil {
        panic("nil contents")
    }
    fmt.Printf("%q", path, treeEntry.Name, string(contents))
    fmt.Println()

    return 0 // continue
}

func main() {
    filename := os.Args[1]

    repository, err := git.OpenRepository(filename)
    if err != nil {
        panic(err)
    }
    defer repository.Free()

    myRepo := MyRepo{repository}

    revWalk, err := repository.Walk()
    if err != nil {
        panic(err)
    }
    defer revWalk.Free()

    // Start out at the head
    err = revWalk.PushHead()
    if err != nil {
        panic(err)
    }

    err = revWalk.Iterate(func(commit *git.Commit) bool {
        defer commit.Free()

        tree, err := commit.Tree()
        if err != nil {
            panic(err)
        }
        defer tree.Free()

        err = tree.Walk(myRepo.foo)
        if err != nil {
            panic(err)
        }

        return true
    })

    if err != nil {
        panic(err)
    }
}

Some errors that come from the above code:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x40f4299]

goroutine 1 [running]:
github.com/libgit2/git2go.CallbackGitTreeWalk(0x438a520, 0x51008e0, 0xc208071e40, 0x0)
    /Users/maleticm/.gvm/pkgsets/go1.4/jack/src/github.com/libgit2/git2go/tree.go:99 +0x79
github.com/libgit2/git2go._Cfunc__go_git_treewalk(0x5100850, 0x0, 0xc208069e40, 0x0)
    /Users/maleticm/.gvm/pkgsets/go1.4/jack/src/github.com/libgit2/git2go/:402 +0x43
github.com/libgit2/git2go.Tree.Walk(0x5100850, 0xc20802c018, 0x5100850, 0xc208069e80, 0x0, 0x0)

runtime: garbage collector found invalid heap pointer *(0xc208069cd0+0x10)=0xc208083e40 span=0xc20807e000-0xc208083800-0xc208084000 state=0
fatal error: invalid heap pointer

runtime stack:
runtime.throw(0x437df83)
    /Users/maleticm/.gvm/gos/go1.4/src/runtime/panic.go:491 +0xad fp=0x7fff5fbfea18 sp=0x7fff5fbfe9e8
scanblock(0xc208069cd0, 0x20, 0x42c12e4)
    /Users/maleticm/.gvm/gos/go1.4/src/runtime/mgc0.c:378 +0x551 fp=0x7fff5fbfeb58 sp=0x7fff5fbfea18
scanframe(0x7fff5fbfec60, 0x0, 0x1)
    /Users/maleticm/.gvm/gos/go1.4/src/runtime/mgc0.c:740 +0x1c2 fp=0x7fff5fbfebc8 sp=0x7fff5fbfeb58

unexpected fault address 0xb01dfacedebac1e
fatal error: fault
[signal 0xb code=0x1 addr=0xb01dfacedebac1e pc=0x40f4299]

goroutine 1 [running, locked to thread]:
runtime.gothrow(0x426ec90, 0x5)
    /Users/maleticm/.gvm/gos/go1.4/src/runtime/panic.go:503 +0x8e fp=0xc208069c40 sp=0xc208069c28
runtime.sigpanic()
    /Users/maleticm/.gvm/gos/go1.4/src/runtime/sigpanic_unix.go:29 +0x261 fp=0xc208069c90 sp=0xc208069c40
github.com/libgit2/git2go.CallbackGitTreeWalk(0x438a520, 0x5009680, 0xc208073e40, 0x0)

After upgrading from go1.3 to go1.4, I'd get a runtime crash most of the time. It crashes for various reasons (nil tree walker callback, invalid heap pointer...). Here is a scaled down version of the code that demonstrates the problem (runtime crashes most of the time). Run this code with an argument pointing at a git repository like git2go. ``` go package main import ( "fmt" "github.com/libgit2/git2go" "os" "strings" ) type MyRepo struct { *git.Repository } func (repository *MyRepo) foo(path string, treeEntry *git.TreeEntry) int { if git.ObjectBlob != treeEntry.Type { return 0 // continue } if !strings.HasSuffix(treeEntry.Name, ".go") { return 0 // continue } blob, err := repository.Repository.LookupBlob(treeEntry.Id) defer blob.Free() if err != nil { panic(err) } if blob.Size() > 10000 { return 0 // continue } contents := blob.Contents() if contents == nil { panic("nil contents") } fmt.Printf("%q", path, treeEntry.Name, string(contents)) fmt.Println() return 0 // continue } func main() { filename := os.Args[1] repository, err := git.OpenRepository(filename) if err != nil { panic(err) } defer repository.Free() myRepo := MyRepo{repository} revWalk, err := repository.Walk() if err != nil { panic(err) } defer revWalk.Free() // Start out at the head err = revWalk.PushHead() if err != nil { panic(err) } err = revWalk.Iterate(func(commit *git.Commit) bool { defer commit.Free() tree, err := commit.Tree() if err != nil { panic(err) } defer tree.Free() err = tree.Walk(myRepo.foo) if err != nil { panic(err) } return true }) if err != nil { panic(err) } } ``` Some errors that come from the above code: ``` panic: runtime error: invalid memory address or nil pointer dereference [signal 0xb code=0x1 addr=0x0 pc=0x40f4299] goroutine 1 [running]: github.com/libgit2/git2go.CallbackGitTreeWalk(0x438a520, 0x51008e0, 0xc208071e40, 0x0) /Users/maleticm/.gvm/pkgsets/go1.4/jack/src/github.com/libgit2/git2go/tree.go:99 +0x79 github.com/libgit2/git2go._Cfunc__go_git_treewalk(0x5100850, 0x0, 0xc208069e40, 0x0) /Users/maleticm/.gvm/pkgsets/go1.4/jack/src/github.com/libgit2/git2go/:402 +0x43 github.com/libgit2/git2go.Tree.Walk(0x5100850, 0xc20802c018, 0x5100850, 0xc208069e80, 0x0, 0x0) ``` ``` runtime: garbage collector found invalid heap pointer *(0xc208069cd0+0x10)=0xc208083e40 span=0xc20807e000-0xc208083800-0xc208084000 state=0 fatal error: invalid heap pointer runtime stack: runtime.throw(0x437df83) /Users/maleticm/.gvm/gos/go1.4/src/runtime/panic.go:491 +0xad fp=0x7fff5fbfea18 sp=0x7fff5fbfe9e8 scanblock(0xc208069cd0, 0x20, 0x42c12e4) /Users/maleticm/.gvm/gos/go1.4/src/runtime/mgc0.c:378 +0x551 fp=0x7fff5fbfeb58 sp=0x7fff5fbfea18 scanframe(0x7fff5fbfec60, 0x0, 0x1) /Users/maleticm/.gvm/gos/go1.4/src/runtime/mgc0.c:740 +0x1c2 fp=0x7fff5fbfebc8 sp=0x7fff5fbfeb58 ``` ``` unexpected fault address 0xb01dfacedebac1e fatal error: fault [signal 0xb code=0x1 addr=0xb01dfacedebac1e pc=0x40f4299] goroutine 1 [running, locked to thread]: runtime.gothrow(0x426ec90, 0x5) /Users/maleticm/.gvm/gos/go1.4/src/runtime/panic.go:503 +0x8e fp=0xc208069c40 sp=0xc208069c28 runtime.sigpanic() /Users/maleticm/.gvm/gos/go1.4/src/runtime/sigpanic_unix.go:29 +0x261 fp=0xc208069c90 sp=0xc208069c40 github.com/libgit2/git2go.CallbackGitTreeWalk(0x438a520, 0x5009680, 0xc208073e40, 0x0) ```

carlosmn commented

2015-01-04 10:35:20 -06:00

(Migrated from github.com)

The newer releases of Go (at least that's where I've noticed it) can make the GC free memory from a variable in the middle of a function if the variable isn't in use in the Go code anymore. It's possible that there are some missing defensive copies.

carlosmn commented

2015-01-09 03:43:38 -06:00

(Migrated from github.com)

It looks like there's a timing aspect to this. Removing the prints lets the program run without crashing.

kron4eg commented

2015-03-20 08:25:28 -05:00

(Migrated from github.com)

I'm getting same panic, EVEN WITH empty callback, without any prints or anything else. Like this:

WalkCallback := func(dirPath string, ent *git.TreeEntry) int {
    return 0
}

rootTree.Walk(WalkCallback)

But my tracebacks are always pointing to this: https://github.com/libgit2/git2go/blob/master/tree.go#L109

err := C._go_git_treewalk(
    t.cast_ptr,
    C.GIT_TREEWALK_PRE,
    unsafe.Pointer(&callback),
)

I'm getting same panic, EVEN WITH empty callback, without any prints or anything else. Like this: ``` WalkCallback := func(dirPath string, ent *git.TreeEntry) int { return 0 } rootTree.Walk(WalkCallback) ``` But my tracebacks are always pointing to this: https://github.com/libgit2/git2go/blob/master/tree.go#L109 ``` err := C._go_git_treewalk( t.cast_ptr, C.GIT_TREEWALK_PRE, unsafe.Pointer(&callback), ) ```

kron4eg commented

2015-03-21 09:55:44 -05:00

(Migrated from github.com)

@carlosmn

It looks like there's a timing aspect to this.

No it's not timing, it's actual problem with git2go code. In 1.4 invalid (contains int for example) go pointers will cause panic during GC. What we are doing here is instructing Go to treat pointer from C (with int) as it was Go's pointer (unsafe.Pointer) which is wrong. We was able get away with this in 1.3 series, but 1.4 is actually have this fixed. This is by design. If we want to operate foreign pointers uintptr should be used.

Here https://github.com/golang/go/issues/9191#issuecomment-66100290 is explained better.

@carlosmn > It looks like there's a timing aspect to this. No it's not timing, it's actual problem with git2go code. In 1.4 invalid (contains int for example) go pointers will cause panic during GC. What we are doing here is instructing Go to treat pointer from C (with int) as it was Go's pointer (unsafe.Pointer) which is wrong. We was able get away with this in 1.3 series, but 1.4 is actually have this fixed. This is by design. If we want to operate foreign pointers uintptr should be used. Here https://github.com/golang/go/issues/9191#issuecomment-66100290 is explained better.

b35li commented

2015-04-10 03:54:12 -05:00

(Migrated from github.com)

@kron4eg

I got the same issue like your comments. my go version is "go version go1.4.1 linux/amd64"
How shall I fix this?

Thanks in advance, it is urgent!!!!

@kron4eg I got the same issue like your comments. my go version is "go version go1.4.1 linux/amd64" How shall I fix this? Thanks in advance, it is urgent!!!!

kron4eg commented

2015-04-10 03:55:12 -05:00

(Migrated from github.com)

@b35li use go 1.3.3, it's OK there.

b35li commented

2015-04-10 04:00:39 -05:00

(Migrated from github.com)

@kron4eg
thanks for your comments.
Is there any other way, this module is one part of big project, it is hard to say switch the go version directly for me.

I am confused if the problem is with git2go, why don't we fix in git2go, while we downgrad the go version to suit it?

@kron4eg thanks for your comments. Is there any other way, this module is one part of big project, it is hard to say switch the go version directly for me. I am confused if the problem is with git2go, why don't we fix in git2go, while we downgrad the go version to suit it?

kron4eg commented

2015-04-10 04:02:51 -05:00

(Migrated from github.com)

I think the problem is how git2go handle C pointers. Maybe I'm wrong, but I don't have time to fix it myself :) So I just downgraded.

b35li commented

2015-04-10 04:08:03 -05:00

(Migrated from github.com)

@kron4eg Got it. thanks very much. I will downgrade it and have a try.

@carlosmn Any comments?

@kron4eg Got it. thanks very much. I will downgrade it and have a try. @carlosmn Any comments?

carlosmn commented

2015-04-10 04:22:04 -05:00

(Migrated from github.com)

@kron4eg if we are using ints as pointers, that's definitely a bug, but I don't know which code you're saying that is doing this.

kron4eg commented

2015-04-10 04:35:10 -05:00

(Migrated from github.com)

@carlosmn I can't say exactly (in other case I'd send PR with fix), but looks like an int pointer returned from libgit2 itself at some point. And Go1.4 runtime see it and crash.

b35li commented

2015-04-10 05:32:18 -05:00

(Migrated from github.com)

@kron4eg Exactly, I have no idea what root cause it is , however, when I downgrade to 1.3.3 it indeed works. wonderful.

@carlosmn if you need any feedback from me, just @ me. Thanks.

@kron4eg Exactly, I have no idea what root cause it is , however, when I downgrade to 1.3.3 it indeed works. wonderful. @carlosmn if you need any feedback from me, just @ me. Thanks.

shinningstar commented

2015-04-18 10:02:16 -05:00

(Migrated from github.com)

I am using 1.4.2 for Tree.Walk testing and it always runtime panic for "nil pointer reference" error.
After printing some log, it shows the callback function pointer(as the payload of git_tree_walk) becomes invalid after a while. It seems that the memory of use provided callback function is recycled by GC.

func CallbackGitTreeWalk(_root *C.char, _entry *C.git_tree_entry, ptr unsafe.Pointer) C.int {
    root := C.GoString(_root)
    entry := _entry
    callback := *(*TreeWalkCallback)(ptr)
    fmt.Println("callback func %v", callback)
    return C.int(callback(root, newTreeEntry(entry)))
}

callback func 0x488c30
ext/pkg/linux_amd64/golang.org/x/tools/refactor/importgraph.a
callback func 0x488c30
ext/pkg/linux_amd64/golang.org/x/tools/refactor/lexical.a
callback func 0x488c30
ext/pkg/linux_amd64/golang.org/x/tools/refactor/rename.a
callback func <nil>
--- FAIL: TestWalk (5.86s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
    panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x5251d5]

I am using 1.4.2 for Tree.Walk testing and it always runtime panic for "nil pointer reference" error. After printing some log, it shows the callback function pointer(as the payload of git_tree_walk) becomes invalid after a while. It seems that the memory of use provided callback function is recycled by GC. ``` go func CallbackGitTreeWalk(_root *C.char, _entry *C.git_tree_entry, ptr unsafe.Pointer) C.int { root := C.GoString(_root) entry := _entry callback := *(*TreeWalkCallback)(ptr) fmt.Println("callback func %v", callback) return C.int(callback(root, newTreeEntry(entry))) } ``` ``` callback func 0x488c30 ext/pkg/linux_amd64/golang.org/x/tools/refactor/importgraph.a callback func 0x488c30 ext/pkg/linux_amd64/golang.org/x/tools/refactor/lexical.a callback func 0x488c30 ext/pkg/linux_amd64/golang.org/x/tools/refactor/rename.a callback func <nil> --- FAIL: TestWalk (5.86s) panic: runtime error: invalid memory address or nil pointer dereference [recovered] panic: runtime error: invalid memory address or nil pointer dereference [signal 0xb code=0x1 addr=0x0 pc=0x5251d5] ```

shinningstar commented

2015-04-19 17:06:19 -05:00

(Migrated from github.com)

I think the problem is that GC has freed the pointer of callback since it doesn't known that it is referenced by C code. So to prevent GC on the pointer of callback, I've add it into a map and delete it after C._go_git_treewalk call finished. The changed code on tree.go is here:

func (t Tree) Walk(callback TreeWalkCallback) error {
    runtime.LockOSThread()
    defer runtime.UnlockOSThread()
    cbmap := make(map[unsafe.Pointer]interface{}, 1)
    ptr := unsafe.Pointer(&callback)
    cbmap[ptr] = callback
    defer func() { delete(cbmap, ptr) }() 
    err := C._go_git_treewalk(
        t.cast_ptr,
        C.GIT_TREEWALK_PRE,
        ptr,
    )   

    if err < 0 {
        return MakeGitError(err)
    }   

    return nil 
}

@carlosmn Do you have any comments on this?
@kron4eg Could you help to test this code on your situation?

I think the problem is that GC has freed the pointer of callback since it doesn't known that it is referenced by C code. So to prevent GC on the pointer of callback, I've add it into a map and delete it after C._go_git_treewalk call finished. The changed code on tree.go is here: ``` go func (t Tree) Walk(callback TreeWalkCallback) error { runtime.LockOSThread() defer runtime.UnlockOSThread() cbmap := make(map[unsafe.Pointer]interface{}, 1) ptr := unsafe.Pointer(&callback) cbmap[ptr] = callback defer func() { delete(cbmap, ptr) }() err := C._go_git_treewalk( t.cast_ptr, C.GIT_TREEWALK_PRE, ptr, ) if err < 0 { return MakeGitError(err) } return nil } ``` @carlosmn Do you have any comments on this? @kron4eg Could you help to test this code on your situation?

shinningstar commented

2015-04-19 17:10:54 -05:00

(Migrated from github.com)

By the way, to reproduce the runtime panic on tree_walk, you should walk a larger git repository to make GC happened during walking. Maybe to reduce the percentage of GC by debug.SetGCPercent() is another way.

pks-t commented

2015-04-20 04:22:39 -05:00

(Migrated from github.com)

It's easy to reproduce the error when walking a rather big tree when starting off the GC via a goroutine. The following example crashes reliably for me on the Linux kernel repository:

package main

import (
    "fmt"
    git "github.com/libgit2/git2go"
    "os"
    "runtime"
    "time"
)

func main() {
    path := os.Args[1]

    repo, _ := git.OpenRepository(path)
    ref, _ := repo.Head()
    commit, _ := repo.LookupCommit(ref.Target())
    tree, _ := commit.Tree()

    go func() {
        time.Sleep(100 * time.Millisecond)
        runtime.GC()
    }()

    if err := tree.Walk(func(dir string, entry *git.TreeEntry) int {
        fmt.Println(entry.Name)
        return 0
    }); err != nil {
        panic(err)
    }
}

It's easy to reproduce the error when walking a rather big tree when starting off the GC via a goroutine. The following example crashes reliably for me on the Linux kernel repository: ``` go package main import ( "fmt" git "github.com/libgit2/git2go" "os" "runtime" "time" ) func main() { path := os.Args[1] repo, _ := git.OpenRepository(path) ref, _ := repo.Head() commit, _ := repo.LookupCommit(ref.Target()) tree, _ := commit.Tree() go func() { time.Sleep(100 * time.Millisecond) runtime.GC() }() if err := tree.Walk(func(dir string, entry *git.TreeEntry) int { fmt.Println(entry.Name) return 0 }); err != nil { panic(err) } } ```

pks-t commented

2015-04-20 05:01:21 -05:00

(Migrated from github.com)

And yes, the error does not occur anymore after the workaround by @shinningstar is applied. Unfortunately it seems as if we don't have any way of doing what we want to do that is completly reliable, as the GC of Go is allowed to copy stacks around, thus causing pointers to become invalid. So if we hand over a Go pointer to a C function and the GC kicks in, causing pointers to become invalid, we're screwed up. As of Go 1.4 handing over Go pointers is not supported (see golang/go#8310).

So I guess the only thing we can do atm is to use the proposed workaround, as it at least seems to make the issue go away, even though it might not prove reliable.

And yes, the error does not occur anymore after the workaround by @shinningstar is applied. Unfortunately it seems as if we don't have any way of doing what we want to do that is completly reliable, as the GC of Go is allowed to copy stacks around, thus causing pointers to become invalid. So if we hand over a Go pointer to a C function and the GC kicks in, causing pointers to become invalid, we're screwed up. As of Go 1.4 handing over Go pointers is not supported (see golang/go#8310). So I guess the only thing we can do atm is to use the proposed workaround, as it at least seems to make the issue go away, even though it might not prove reliable.

carlosmn commented

2015-04-21 05:21:39 -05:00

(Migrated from github.com)

If the runtime is going to arbitrarily make all of our pointers invalid, we'll have to write an indirection layer, as we keep our pointers long-term.

Whether this is due to the stack moving or overeager GC, we will need a more comprehensive method for converting pointers between libgit2 and the Go code.

If the runtime is going to arbitrarily make all of our pointers invalid, we'll have to write an indirection layer, as we keep our pointers long-term. Whether this is due to the stack moving or overeager GC, we will need a more comprehensive method for converting pointers between libgit2 and the Go code.

carlosmn commented

2015-04-21 06:56:01 -05:00

(Migrated from github.com)

I'm currently unable to trigger this error, even when running against the linux repo. I have however created a branch cmn/pointer-indirection which should work around both an eager GC and the stack moving issue for this particular code. The rest of the callbacks should also get this treatment.

I'm currently unable to trigger this error, even when running against the linux repo. I have however created a branch `cmn/pointer-indirection` which should work around both an eager GC and the stack moving issue for this particular code. The rest of the callbacks should also get this treatment.

shinningstar commented

2015-04-22 10:58:22 -05:00

(Migrated from github.com)

@carlosmn You could see issue#10303 in golang, maybe it could help.

@carlosmn You could see [issue#10303](https://github.com/golang/go/issues/10303) in golang, maybe it could help.

pks-t commented

2015-04-23 03:09:34 -05:00

(Migrated from github.com)

@carlosmn: I've modified the code to use your pointer indirection branch (see pks-t/git2go/pointer-indirection. Seems to work after fixing a bug in the NewHandleList function, at least my code snippet posted above does not error out anymore.

If this is the way we want to go I'll also convert other functions that use Go callbacks in C code.

@carlosmn: I've modified the code to use your pointer indirection branch (see [pks-t/git2go/pointer-indirection](https://github.com/pks-t/git2go/tree/pointer-indirection). Seems to work after fixing a bug in the NewHandleList function, at least my code snippet posted above does not error out anymore. If this is the way we want to go I'll also convert other functions that use Go callbacks in C code.

shinningstar commented

2015-04-23 22:13:10 -05:00

(Migrated from github.com)

@pks-t @carlosmn How about using the empty(NOP) function to escape callback to heap, just like go runtime team using? It seems that cgo team will use this method in Go1.5 to escape Go world pointer when passing to C world.

//go:noescape
func use(unsafe.Pointer) {}

func (t Tree) Walk(callback TreeWalkCallback) error {
    runtime.LockOSThread()
    defer runtime.UnlockOSThread()

    err := C._go_git_treewalk(
        t.cast_ptr,
        C.GIT_TREEWALK_PRE,
        unsafe.Pointer(&callback),
    )   
    use(unsafe.Pointer(&callback))

    if err < 0 {
        return MakeGitError(err)
    }

    return nil 
}

@pks-t @carlosmn How about using the empty(NOP) function to escape callback to heap, just like go runtime team using? It seems that cgo team will use this method in Go1.5 to escape Go world pointer when passing to C world. ``` go //go:noescape func use(unsafe.Pointer) {} func (t Tree) Walk(callback TreeWalkCallback) error { runtime.LockOSThread() defer runtime.UnlockOSThread() err := C._go_git_treewalk( t.cast_ptr, C.GIT_TREEWALK_PRE, unsafe.Pointer(&callback), ) use(unsafe.Pointer(&callback)) if err < 0 { return MakeGitError(err) } return nil } ```

pks-t commented

2015-04-24 00:52:59 -05:00

(Migrated from github.com)

Well, for my part I wouldn't really want to rely on workarounds that are not officially documented. For now it is explicitly stated that no Go pointers should be passed into the C world, so I'd like to adhere to that and use the pointer indirection branch of @carlosmn insetad.

shinningstar commented

2015-04-24 01:40:32 -05:00

(Migrated from github.com)

@pks-t OK, the pointer indirection is fine. I could patch my project locally and wish it merged into main branch soon.

pks-t commented

2015-04-24 04:10:26 -05:00

(Migrated from github.com)

@shinningstar I've already converted most of the code to use pointer indirection, see #196.

Sign in to join this conversation.