Outlier
An edtech platform where the database had become the bottleneck. I profiled the slow paths, rewrote the queries behind the busiest endpoints, and built an account-merge tool that remaps student records across 12+ tables without losing history.
Category
Back-End
Year
2024
Duration
9 months
Team
15 engineers
Overview
Outlier runs dual-enrolment courses for high-school students. I joined the backend team for performance work: the database had become the constraint on most of the platform, and several long-standing data problems had never been picked up. The work split three ways — profiling and rewriting the slowest execution paths, fixing the query shapes the ORM was generating badly, and building tooling for a duplicate-account problem that had been handled by hand until then.
The problem
- A core background job was taking 3 minutes per run and blocking downstream processes during peak hours.
- Several API endpoints had no query optimization — full table scans on large datasets, no indexes on join columns.
- Duplicate student accounts were accumulating with no clean merge path, and manual cleanup was error-prone.
- GCP integrations were failing silently in edge cases, causing downstream data inconsistencies that were hard to trace.
The solution
- Profiled the cronjob execution path, found N+1 query patterns and unnecessary in-memory sorting, and rewrote it with batched queries and a single-pass sort.
- Added composite indexes on the most-queried columns, rewrote several ORM-generated queries as raw SQL where the query planner was making bad choices.
- Wrote a migration-style merge script with a dry-run mode: it maps every foreign-key relationship first, then remaps in dependency order inside a transaction, so a failure rolls back cleanly.
- Added structured logging and alerting on GCP integration failure paths, then fixed the root-cause race conditions in the retry logic.
Outcomes
- Cronjob execution time: 3 minutes → 45 seconds (75% reduction)
- Average query time reduced by 40% across high-traffic endpoints.
- Student account merge tool handled 12+ table relationships with zero data loss.
- System downtime reduced by 15% after the GCP integration fixes.
- Test suite with Jest + Supertest caught multiple regressions before production.
My role
Backend Developer
Node.js, Express.js, Redis, JavaScript, PostgreSQL, Git