[WIP/RFC] Pointer indirection #196
Loading…
Reference in New Issue
No description provided.
Delete Branch "pointer-indirection"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This pull request fixes issues with Go callbacks that are handed over to C functions. As of Go 1.4 it is not possible to hand over Go pointers into C code as the garbage collector is allowed to copy around the stack, causing pointers to become invalid (see golang/go#8310 and golang/go#10303 for more information).
@carlosmn proposed an interface HandleList for pointer indirection which provides a global map of data that is to be tracked. Instead of handing in pointers we now instead hand over an index that points to the corresponding entry in this map.
This PR adds the HandleList implemented by @carlosmn, fixes some issues with it and adjusts code that hands over Go pointers into C code to use handles instead. Still missing is the remote.go code, as I am running into some error there ('runtime: garbage collector found invalid heap pointer *(0xc208063cd0+0x18)=0x1 s=nil') that I've not been able to fix.
This PR also fixes #163.
I guess it makes sense to not fix the remotes code but instead wait for libgit2/libgit2#3066 to be merged, as the interface will change anyway. The rest could be merged independetly of this, though.
I've changed the code to panic when no treewalk/submodulevisitor callback could be found. The
indexMatchedPathCallback
payload may benil
, though, as it is allowed to callAddAll
without a callback.Any comments regarding this PR?
I tested "git diff" with go1.4. It solves the panic caused by GC.
There's still callbacks where we check whether the pointer is valid. If we do not find the handle in the list, there is no way we can continue running the program, as the state has been corrupted. We should either assume that the callback is correct, or panic if the retrieval fails.
@pks-t
The latest commits broke it again.
@taylorchu Eh, you're correct. Seems as if I worked on an outdated branch of mine.
Interestingly enough the problem was not caused by an outdated copy but by a change of the GC behavior. When the GC kicks in it inspects uintptr values, causing invalid dereferences due to us storing callback data by arbitrary uinptrs. I have no idea why the issue did not occur previously, but it's fixed now (at least for me). @taylorchu could you please verify the fix?
Short bump. Any remarks left?
Hi,
I believe there is a relatively minor (and fixable) problem with the code change in this PR (I got to this PR from commit
1bd338af5e
, which touches the relevant code). I have found a reproducible test case that causes a panic because the Go garbage collector triggers and removes something that shouldn't be removed (because there are no references to it in the Go world, only in the C world). (/cc @slimsag, thanks for reviewing my findings and suggesting a fix.)I will create an issue with some steps you can follow to reproduce the issue.
I also have an MVP fix for the issue, that reliably resolves the problem. There's more than one way of doing it, so I welcome review and suggestions to change it, but I believe the general idea of the fix is sound, or at least in the right direction (basically, to keep a reference to the value in Go world memory so that it doesn't get GCed). I will create the PR and reference the issue.
I've made that issue with a reproducible failing test case in #218, and a PR that fixes it in #219.