01. Observed production placement
The measured infer graph assigned 2,090 operators to ANE and 79 to CPU (96.4% versus 3.6%). The CPU-assigned operations were in the language-model-head argmax/output tail. This distinction matters: the layer stack and KV machinery can be ANE-resident without the output selection tail being ANE-resident.
| State primitive | ANE | CPU | Classification |
|---|---|---|---|
slice_update | 56 | 0 | ANE |
read_state | 57 | 0 | ANE |
write_state | 56 | 0 | ANE |
slice_by_index | 225 | 0 | ANE |
The stateful pattern used padded widths, sequence geometry and graph structure compatible with compiler lowering rules. A failing isolated slice_update micrograph was not evidence that state writes require CPU execution in the production graph.
02. Gather is a type contract
Core ML's ANE lowering accepted FP16 data/output gather tables with integer indices. Floating-point gather indices were not a valid substitute: FP16 cannot represent every integer precisely beyond 2,048, so forcing token IDs through that type creates both compilation and correctness hazards.
// Conceptual lowering contract
position_fp16
└─ cast(to: uint16)
└─ gather(fp16_rope_table, index: uint16)
└─ fp16_rope_embedding
// NOT an ANE lowering contract
gather(data: fp16, index: fp16)Model-vocabulary token lookup can stay outside the ANE computation boundary while ANE receives FP16 embedding tensors. The implementation treats this as data movement and does not mislabel it as an ANE compute operation.
03. A misleading compiler error
One graph failed in CreateBnnsGraphProgramFromMIL with Invalid sentinel in blob_metadata. The message was ambiguous: an unlowerable operator or an external-weight offset landing in the wrong bytes could produce a related failure. The correct diagnostic was a walk of every BLOBFILE offset referenced by MIL.
- FILE HEADER
- 64 bytes
- PER-BLOB HEADER
- 64-byte metadata, sentinel 0xDEADBEEF
- DATA
- 64-byte aligned
- REFERENCES
- Offsets target metadata headers, not blob payloads
Appending new blobs without rewriting all existing offsets preserves the original graph's references. A “clean” MIL rewrite is insufficient if new weight references never received correctly indexed backing blobs.
04. Fidelity is not placement
After RoPE and blob-reference corrections, the hand-authored LUT4 28-layer model measured 16.8033 perplexity on WikiText-2. It differed from a contemporaneous Apple Core AI comparison (14.79 macOS) by approximately 2.01 perplexity points; that reference is a distinct model/configuration and does not establish a general quality penalty of four-bit quantization.
05. What is actually supported
2,090/79 operations and the four stateful-op counts were obtained using MLComputePlan on the named compiled artifact. They do not prove zero CPU work in the total inference stack, privileged access to ANE firmware, or transferability to a different Core ML build.
Source records: ane-re/docs/FINDINGS.md (§2, §6, §11–§12); ANE-RE ledger D210, D236, D257, D262, D263. Source and model artifacts remain private; this page publishes a bounded technical account.