Drift and Trust

Acquiresignal in

Why a decoder goes stale overnight

Put an electrode array in motor cortex and you do not get a wire into a thought. You get a few hundred channels of voltage, each one picking up whatever neurons happen to be near its tip, and a decoder — a statistical model — that learns to map those voltages onto an intention: move left, click, hold.

The hard part is that the mapping will not stay put. The array shifts by micrometres as the brain moves against the skull. Tissue response builds up around the electrodes and changes what each channel can hear. Individual neurons drop out of range and new ones drift in, so the channel that meant left on Tuesday is listening to a different cell on Wednesday. And the cortex itself is learning — the user is adapting to the decoder while the decoder is trying to hold still.

So the decoder was rebuilt, constantly. A calibration block at the start of each session: follow the target, think the movement, let the model refit. Ten minutes of your day, every day, before the interface was worth using. The field has a benchmark named after exactly this problem — FALCON, Few-shot Algorithms for Consistent Neural decoding — which exists because cross-session stability, not raw accuracy, is what stands between a lab demo and a device someone lives with.

Pretrainshared prior

What fifty thousand hours bought

The move Neuralink reported in October is the one that has worked everywhere else in machine learning, applied to neural data. Instead of fitting each person’s decoder from scratch on labelled examples — here is the signal while they intended left — they used the accumulated recording from trial participants, over 50,000 hours of it, to pretrain self-supervised neural encoders. The encoder learns the structure of cortical activity in general, from a corpus with no labels attached. Your own decoder then becomes a small adaptation on top of a prior that already knows what brains look like.

This is the same curve that made language models useful: pretrain broadly on cheap unlabelled data, adapt narrowly on the expensive labelled kind. Here the expensive labelled kind is minutes of a person’s attention, which is why the economics matter so much. The reported result is that for some participants the calibration block fell from roughly ten minutes every day to roughly ten minutes a week, and some decoders stayed usable for weeks instead of days.

Throughputbits out

11.32 bits per second, in context

Alongside the calibration result came a throughput record: 11.32 bits per second of cursor control. Bits per second here is a Fitts-law style measure — how much target-selection information the user can push through the channel per second — and it is the right axis to compare interfaces on, which makes it worth putting next to the alternatives.

The honest comparison is a mixed one. Against the best published intracortical cursor control in people with paralysis, 11.32 is a large jump. Against talking, it is not close.

The argumentcase & objection

Does any of this lead to integrated machine intelligence?

This is where the interesting disagreement is, so it is worth stating the strong version of the case before taking it apart.

The case

Bandwidth has been the binding constraint on human–machine integration since the beginning, and it is the constraint that just moved. Every interface we have — keyboard, touchscreen, voice — is a narrow pipe bolted onto a wide brain, and the pipe, not the brain, is what sets the ceiling on how tightly a person and a machine can work as one system. A channel that went from ~4 bit/s to ~11 bit/s in a decade is a channel that is actually improving.

Personalisation is the unlock, and the new result shows the per-person part can be cheap. That matters more than it sounds. If every individual needed a bespoke model trained on hours of their own labelled data, integration would never scale past a research cohort. A shared prior plus ten minutes of you is a different economic object entirely — it is the difference between a bespoke prosthesis and a device with a setup wizard.

And the loop is genuinely two-sided. The user’s cortex adapts to the decoder while the decoder adapts to the cortex — co-adaptation is a documented, central phenomenon in this field, not a metaphor. Two learning systems in a closed loop is a qualitatively different thing from a peripheral, and it is the most defensible sense in which these devices “integrate” with a person at all.

The objection

Three things bite, and they bite hard.

11.32 bit/s is a cursor. It is not a new sense, a thought channel, or language. It is target selection — roughly a third of the rate at which you can simply talk, achieved by a surgically implanted array, and it is a magnificent result for someone who cannot move or speak. Assistive-technology breakthroughs and intelligence breakthroughs are different categories, and the AGI framing quietly borrows the emotional weight of the first to make a claim about the second.

A decoder is a measuring instrument. Sharpening an instrument does not make the thing it measures smarter. Reading motor intent from M1 at high fidelity is a magnificent feat of signal processing and tells you approximately nothing about how to build general intelligence. The argument “better interface to a brain, therefore better intelligence” has a missing step where its whole weight should be.

“Integrated SGI” has no operational definition, which means it cannot be wrong — and a thesis that cannot be wrong is not a strong one, it is an unfalsifiable one. This is the failure mode that this field’s loudest claims keep walking into, and it is avoidable.

Where it lands

Bandwidth-plus-co-adaptation is a real research programme, and the calibration result is a real advance inside it. “Therefore AGI” is unearned — not disproven, unearned, which is a different and more fixable problem. The repair is to say what would earn it, and that list is at the end of this piece.

An unfalsifiable thesis is not a strong one. The repair is to say what would earn it.

Shipattack surface

The update is the attack surface

Here is the part that should interest anyone who works in security, and it follows directly from the calibration result rather than from any speculation about minds.

When every decoder was fit from scratch, on-device, from your own session data, the model was in an important sense local. There was not much to ship. A shared pretrained encoder changes that: the thing that interprets your motor cortex is now an artefact built elsewhere, by someone else, on a corpus you have never seen, and delivered to hardware inside your skull. That is a software supply chain, and it has all the properties of one.

The existing literature on this is not thin, and it is not reassuring. Adversarial and backdoor attacks against EEG-based BCIs are demonstrated, not hypothetical: small crafted perturbations flip classifier outputs, and poisoned training data installs behaviour that stays dormant until a trigger appears. The surveys on BCI security have been flagging the implanted-device case — where a firmware or model change is a patient-safety event, not a data-integrity one — for years.

Code signing proves where an artefact came from. It says nothing at all about whether it is benign.

Verifythe proposal

A changelog you can actually check

I build a reverse-engineering toolkit called repX — an open, zero-dependency Rust workspace for static binary triage. It is at v0.1 and its README carries a “Reality check” table separating what runs today from what is a research goal, which is the standard I am going to hold this section to as well.

What repX does today is narrow and specific: hand it a blob and it reports container format, CPU architecture, word size, endianness, size, Shannon entropy, magic bytes, and a content hash — without executing it. The hash primitive, FNV-1a-64, is implemented twice: once in hand-written AArch64 and x86-64 assembly via global_asm!, and once as a pure-Rust reference, with a test asserting the two agree. There is a typed control-flow-graph scaffold with a Graphviz exporter, and trnX, a node-registry skeleton for distributing analysis across machines.

I want to argue that this check set — not the tool, the shape of it — is close to what a decoder update needs, and then be precise about where it falls short.

Four things that transfer

• Hash it twice, with independent implementations. repX asserts an assembly primitive against a Rust reference because a single implementation is a single point of trust. A decoder artefact's digest should be independently reproducible for the same reason — one party's hash of their own artefact is an assertion, not evidence.
• Triage the shape before running it. Format, architecture and entropy tell you whether a blob looks like what it claims to be, without executing a line. An implant should be able to ask the same question of an incoming model: are these weights structured the way weights are, or is there a region with the entropy profile of something else entirely?
• Diff the structure, not the release notes. A CFG diff between two versions is a changelog that cannot lie about itself: these blocks changed, these edges appeared. Model artefacts need the equivalent — a structural delta a reviewer can bound, rather than a vendor's prose summary of their own update.
• Corroborate across independent parties. trnX's premise is many nodes computing the same analysis and agreeing. One attestation is a claim; many mutually independent agreeing attestations is evidence. This is the piece that matters most here and the piece that is least built — see below.

And the part that is genuinely new: a held-out probe

Byte-level integrity is not enough, because the attack that matters most passes it. Imagine a model that is correctly signed by the real key and was fine-tuned on poisoned data. Its signature verifies. Its hash reproduces. Its structure barely changed. Nothing about its provenance is false. It is simply wrong in a way someone chose.

What catches that is behaviour, not bytes. Keep a small, private, per-person probe set: recorded neural segments with known intended outputs, held out and never used in training. After any update, replay the probe through the new decoder on the device and compare the output signature against the previous version’s. A benign retrain moves it within a characterised band. A poisoned one moves it outside, or moves it on exactly the inputs that were targeted.

This is the defensible version of putting brain data in the changelog. Not a scan pasted into a release note — a behavioural fingerprint of the decoder, measured on your own neural data, recomputed every time the model changes.

Three reasons this is harder than it sounds

The probe set is the most sensitive artefact in the system. It is a stored record of your neural responses, kept specifically so it can be replayed. Compromise it and an attacker has both an authentication oracle — there is already a granted US patent on authenticating people from neural-activity biometric templates — and a tailored objective function for building an adversarial example aimed at one individual.

Publishing the defence publishes the target. An attacker who knows the probe distribution can fine-tune to pass it. This is the exact shape of the backdoor results in the EEG literature: behave correctly on the evaluation distribution, misbehave on the trigger. A probe set is a test, and tests can be taught to.

Drift and tampering are the same signal. This is the deepest problem, and it is handed to us by the very result this piece opened with. Neural signals drift legitimately, substantially, and in a person-specific way — that is why recalibration existed. A detector that fires on “the decoder’s behaviour changed” has to separate ordinary nonstationarity from deliberate interference, in a system whose headline finding is that ordinary nonstationarity is large. I do not think that discrimination problem is solved, and I am not going to pretend the band in Figure 5 is easy to characterise.

Auditable is a precondition for safe. It is not a synonym for it.

So I will not tell you that verifiable provenance makes brain implants extremely safe. A networked device wired into motor cortex, receiving models built from other people’s data, is not going to be extremely safe in any sense a security engineer should sign their name to. What it can be is auditable by someone other than its manufacturer — and that is worth building, because it is the precondition for every stronger claim anyone wants to make later.

Proposalfalsifiable

What would actually earn the claim

The integration thesis is unearned rather than refuted, so here is the list that would earn it. These are ordered by dependency — each one is roughly a precondition for the next being meaningful — and every one of them is falsifiable, which is the point.

1. 01Bidirectional throughput at comparable rates.Writing to cortex is far behind reading from it. Integration in any strong sense needs a return path with stable, learnable percepts at bit rates in the same order as the 11.32 going the other way. Until then this is a very good output device.
2. 02Transfer beyond motor cortex, across people.The pretraining result should extend from movement intention to language or abstract representation, and the shared prior should carry across subjects. Motor decoding is the easy case — it has a clean behavioural label to train against.
3. 03A prior that improves when a new person joins.If each additional participant measurably improves everyone else's decoder, there is real accumulating cross-subject structure. If the corpus only helps the person it came from, this is personalisation, not a foundation.
4. 04Closed-loop gains after the user stops learning.Hold the user's practice constant and keep improving. That separates “the machine half is carrying new load” from “the human got good at the device”, which is the confound sitting under every co-adaptation claim.
5. 05An update path auditable by a third party.Independent reproducible digests, bounded structural diffs, on-device behavioural probes, and a root of trust outside the manufacturer's signing infrastructure. Without this the rest is a device you have to take on faith, and nobody should put a decoder they cannot audit inside their skull.

Item 5 is the one I can actually work on, which is why it got the longest section. The others I will be watching for — and the first group to publish a decoder whose shared prior demonstrably improves when a stranger enrols will have done something far more interesting than a throughput record.

Sourcesfact-check

Provenance & fact-check

Every number in this piece with where it came from and what kind of evidence it is. The last two rows are claims I encountered while researching and could not verify from a primary source; they are excluded from the body and listed so nobody repeats them on my account.

Reading list

• Pandarinath et al., “High performance communication by people with paralysis using an intracortical brain–computer interface” — eLife 2017. The prior benchmark, and still the clearest write-up of how bitrate is measured here.
• The FALCON benchmark — Few-shot algorithms for consistent neural decoding. Exists because cross-session stability is the field's real problem.
• “Security in Brain-Computer Interfaces: State-of-the-Art, Opportunities, and Future Challenges” — ACM Computing Surveys 54(1). The survey to start from.
• “Adversarial Filtering Based Evasion and Backdoor Attacks to EEG-Based Brain-Computer Interfaces” — The attack shape that motivates behavioural probing.
• “Adversarial Vulnerabilities of Neural Biomarker Identification Systems” — Why a stored neural template is a liability as well as an asset.
• Coupé, Oh, Dediu & Pellegrino, “Different languages, similar encoding efficiency” — Science Advances, 4 September 2019. Where ~39 bit/s comes from.
• repX — The toolkit, its roadmap, and its reality-check table.