Colored-pencil caricature portrait of Tamir Yirga

Tamir Yirga

Software engineer building high-performance data infrastructure and distributed systems. I primarily write Rust, with a growing interest in ML inference systems. Currently finishing my MS in Computer Science at Northeastern.

Projects Showcase

A selection of the personal and course projects I have most enjoyed building. Every number on this page comes from a run I can point to, and where I do not have a number, I say so.

The goal of this project was to understand exactly what happens between a model file sitting on disk and a token appearing on screen, by writing every layer of it myself. That meant a GGUF parser, the tensor handling, the attention mechanism, the sampling, and above all the matrix multiplication kernels which the whole thing lives or dies by.

A 7B model in float32 is roughly 24 GB, so the weights are memory-mapped rather than read into a buffer, and quantizing them to INT8 brings that down to 6 GB. Decoding a single token is one row of activations against a full weight matrix, which makes it bound by memory bandwidth rather than by arithmetic; at INT8 the weight block fits in L2 cache and at float32 it does not. Crossing that cache threshold is worth 21x, which is why the square-matrix benchmarks people usually quote, at around 2x, miss the effect completely.

Every kernel was custom written and benchmarked, and not every rewrite paid off. Cache-tiling the inner loop and adding NEON intrinsics helped at small matrix sizes, but gave nothing back at 1024 square, where it ran at 22.2 Gelem/s against 23.3 for the naive scalar version; the tiled loop structure defeated LLVM's auto-vectorization on ARM. The gain at that size came from Rayon row-parallelism instead, which took the same shape to 157.9.

Generating a token runs this shape once per layer, every layer, every token. The INT8 weight block fits in L2 cache and the f32 block does not, and that cache residency is where the speedup comes from rather than any change in instructions.
f32 GEMM at 1024 square, 10 cores
157.9 Gelem/s
Naive scalar version of the same kernel
23.3 Gelem/s
Tests passing across kernels, cache and sampling
873

These are Criterion benchmarks on an M1 Pro, and the memory footprint is read out of the running process rather than calculated on paper. They are kernel and memory numbers though. A full 7B generation run is still unfinished, so there is no tokens per second figure here yet.

The goal of this project was to stop treating the CAP theorem as something you read about, and to build both sides of it in one codebase. Most systems pick availability or consistency and then describe the tradeoff in their documentation. I wanted the difference to be something I could demonstrate on demand.

Everything below the replication layer is shared between the two paths: the same write-ahead log, the same LSM storage engine with its MemTable, SSTables and compaction, the same Bloom filters. Only the protocol above it differs. On top of that I wrote a linearizability checker modeled on the ones Jepsen uses, and pointed it at both paths under injected partitions. Raft comes back linearizable. The quorum path comes back with a witness, a specific read which no valid ordering can explain. Because the storage engine underneath is identical, the protocol is the only thing which could have produced the difference.

The distinction the checker makes precise is the one which is easiest to get wrong on paper. W + R > N gives quorum overlap, but it does not give linearizability. Overlap only guarantees that a read set and a write set intersect. Linearizability additionally requires that every operation appears to take effect at a single instant between its start and its end, in an order consistent with real time. The quorum path satisfies the first condition and fails the second, and the checker returns the specific read which proves it.

Grafana dashboard showing operations per second, p99 latency, quorum failures and repairs across five LedgerKV nodes while one node is killed and restarted Open full size
Grafana reading Prometheus off all five nodes during that run. I killed node2 at 15:14 and brought it back at 15:16: its line drops out, the other four carry the load, and quorum failures stay flat at zero throughout. The two p99 spikes are JVM warm-up and node2 restarting cold, not the kill itself.
Steady-state p99 latency, p50 was 2 ms
15 ms
Time node2 spent dead mid-run
90 s
Tests passing in CI, linearizability checker included
361

This is one run on one cluster, scraped from each node's /metrics endpoint. The checker is a bounded bug finder rather than a proof: a linearizable verdict only means that no counterexample was found in the history it examined, whereas a non-linearizable verdict is definitive because it comes with a witness. Hinted handoff and read repair are both implemented, but I have not measured how long a recovered replica takes to converge, so there is no number for that here.

Lift-ride events arrive over HTTP, go onto a queue, and are written into DynamoDB with Redis caching the reads. In front of all of that sits a token-bucket admission controller which samples queue depth every 200 ms and steers with AIMD. Backing off is deliberately far more aggressive than recovering: a severe backlog halves the admitted rate straight away, while recovery only adds ten permits a second. If the broker cannot be reached at all, the controller now holds its current rate, because the original version read silence as an empty queue and opened the gates at exactly the wrong moment.

The controller is also fleet-aware. Each replica heartbeats into a Redis sorted set and divides the admission floor by the number of live replicas it can see, so adding a replica does not multiply the rate the cluster admits in total. Scaling up takes effect immediately, but scaling down needs three consecutive confirmations, otherwise a Redis restart could convince a replica it was running alone and hand it the entire budget.

The load harness needed rebuilding before any of these numbers meant anything. The original was closed-loop, which understates latency under load by a factor of 9.5 on this workload. Rebuilding it open-loop, and running every A/B comparison in both orders so that ordering bias could be ruled out, is what makes the remaining figures worth quoting at all.

The same system measured two ways. A closed-loop harness waits for each response before sending the next request, so it slows down alongside the system it is measuring and the queueing delay never lands in the numbers. This is coordinated omission. Charging latency from when a request was due, rather than from when it actually went out, is the fix.
The AWS run: 200,000 requests, every one returning 200
4,309 req/s
Distinct-count write requests against the DynamoDB counter
25,163
Same workload, once HyperLogLog took over the counting
9,635
Defects documented, each one pinned by a regression test
38
Decision records, rejected alternatives included
11

These numbers come from two different setups. The original ran on AWS, a Spring server on EC2 with RabbitMQ and DynamoDB behind it, and the one report I kept from that build records 200,000 requests at 4,309 req/s with every response a 200. Everything after the rebuild ran against LocalStack on a single laptop, where LocalStack is the binding constraint in every run, so read those throughput figures as a property of the test rig rather than of the architecture. Peak offered load in that sweep was 4,376 req/s, above which the controller shed hard. A batching result which looked like a 16% gain did not survive swapping the run order, so it is not on this page. The CI workflow is written but has never actually run.

My responsibility was the data layer end to end: the PostgreSQL schema, eight Flyway migrations, and the Docker Compose stack the rest of the team developed against. Clinical events vary in shape depending on their type, so they live in a JSONB column with a GIN index over the payload, and a partial index keeps soft-deleted rows out of the paths every other query uses. Deletes are soft throughout so the audit trail survives, which matters for anything touching a clinical record.

The part which mattered most was not code. Three teams were building against the same records, so I wrote the single document defining what a clinical event actually is: every enum lowercase, every field named exactly, and the required fields listed per type. Blood pressure gets systolic and diastolic, never a generic value field. A named drug is always a medication and never a procedure, even when it is injected. That document gave the LLM extraction layer an exact contract to target, and stopped three teams inventing three different shapes for the same record.

Alongside the data layer I acted as integration reviewer for the team, reviewing 14 pull requests and blocking 11 of them on line-level findings: a CascadeType.ALL which would have cascade-deleted patient data, double-encoded JSONB corrupting every API response, a Float and Double mismatch across two concurrent branches, and contradictory prompt instructions producing non-deterministic extraction output.

vital_signvital_type, then per type. Blood pressure needs systolic and diastolic. Heart rate carries no unit field because the unit is fixed.
symptomsymptom. Onset is a timestamp, duration is a length of time, and they are not interchangeable.
mental_statestate, one of twelve. No severity field on this type, and nothing outside the list.
medicationmedication_name and status, always. The field is dose_amount, not dose or dosage.
procedureprocedure_name and status.
allergyallergen and allergy_type. Severity describes a past reaction, criticality describes future risk.
otherdescription.
The required fields for each event type, taken from the schema spec. Pinning down the field names and the enum casing is what kept three teams building the same record rather than three variations on it.
Flyway migrations behind the schema, seven of them mine
8
Containers in the Compose stack I assembled
6
Mental state values the spec allows, up from five
12

This was a team project, so the lines matter. The schema, the migrations, the event spec and the Compose stack are mine. The Python extraction consumer and the architecture diagrams are other people's work, and so were the speech-to-text vendor comparison and the model-selection decision. Nobody measured throughput or latency on this build, so there are no performance numbers to show.

The aim was to compare four convolutional architectures, ResNet-50, ConvNeXt-Tiny, EfficientNet-B3 and MobileNetV3, on the same hardware. The harder problem turned out to be the measurement rather than the comparison. MPS and CUDA queue their work asynchronously, so a timer wrapped around a forward pass measures how long it took to hand the kernels to the GPU rather than how long the GPU spent running them.

The protocol the harness settled on is ten untimed warmup passes to absorb Metal shader compilation and let the allocator settle, then a device synchronize so the clock starts on a quiet device, then a hundred timed passes with a synchronize after each one rather than only at the end. It reports mean, standard deviation, min and max, and derives throughput from the mean instead of counting it separately. None of that is complicated, but getting it wrong is easy and the resulting numbers look perfectly plausible.

01Warm upTen untimed passes. Absorbs Metal shader compilation and the allocator settling.
02SynchronizeWait for the queue to actually drain, so the clock starts on a quiet device.
03Time 100 passesperf_counter around each one, synchronizing after every call rather than at the end.
04Report the spreadMean, standard deviation, min and max. Throughput is derived from the mean, not counted separately.
The measurement protocol. Skip the synchronize and a model can look ten times faster than it really is, which is the failure mode this harness exists to avoid.
Architectures through the same harness and hardware
4
Warmup passes before the clock starts
10
Timed passes per measurement
100

This was a three-person course project and the architecture comparison was my track. The harness writes its results into a gitignored directory, so there is nothing committed which can be quoted here. Rerunning it and publishing the full table is the obvious next step.

The point of the exercise was to implement the pipeline directly instead of calling it out of OpenCV, so the thresholding, blur and morphology are all written from scratch. Each frame is thresholded using ISODATA over a 6.25% random pixel sample, which keeps that step cheap enough to run on live video, then segmented into connected regions, and each region is rotated onto its axis of least central moment so that orientation stops mattering.

Regions are then described in one of two ways. The first is seven features chosen to survive rotation and scale: how much of its oriented bounding box the region fills, aspect ratio, the first three Hu moments, compactness and extent. The second flattens the region to a 64 by 64 patch and projects that 4,096-pixel vector onto 20 principal components. Matching happens in those 20 dimensions, which is why a single image is enough to teach it a new object with no training run at all.

01ThresholdISODATA over a 6.25% random pixel sample, so the cutoff tracks the scene without reading every pixel.
02Segment and orientConnected regions, each rotated onto its axis of least central moment.
03DescribeSeven invariant features, or a 64 by 64 canonical patch for the eigenspace path.
04ClassifyNearest neighbor under scaled Euclidean distance, with a cutoff that lets it answer "unknown".
Every stage runs per frame on live video. The 6.25% pixel sample is what keeps the thresholding cheap enough to do that.
Principal components, down from 4,096 pixels
20
Rotation- and scale-invariant features per region
7
Pixels sampled to estimate the threshold
6.25%

There is no benchmark committed for this one. Accuracy figures exist in the old write-up, but they sit in an example output block rather than in a recorded run, so they are not repeated here. There is no timing code in the project either, so there is no honest frame rate to quote. The demo video is the real evidence.

Work Life

Spark 4.1.1 TIME data types in Rust Arrow-native PySpark UDF framework 15x throughput speedup via zero-copy Arrow transfer https://lakesail.com/
RAG text-to-SQL at 98% accuracy Analytics platform with AI chat Self-maintaining pipeline for 700M+ records https://decanaria.com/
8 e-commerce platforms for clients with $50M+ annual revenue Platform migration with 60% legacy refactor Reduced cart abandonment by 18% https://scandiweb.com/
Non-custodial wallet for 7,000+ users Real-time sync with Firestore + WASM 60% faster wallet sync via batched UTXO filtering https://github.com/Endubis-Solutions/

Education

GPA 4.0/4.0 Research Assistant (LLM Evaluation) Teaching Assistant (Scalable Distributed Systems, Computer Vision) Key coursework: Distributed Systems, ML, CV, Algorithms

Certifications

Featured Writing

View all writing