System Design for Mid-Level Engineers: Designing a URL Shortener from Scratch

If you've been asked to "design a URL shortener" in a system design interview, you're not alone — it's one of the most common questions asked at companies ranging from startups to FAANG. It looks deceptively simple on the surface (just map a long URL to a short one, right?), but it actually touches on nearly every core system design concept: capacity estimation, database schema design, encoding algorithms, caching, and horizontal scaling.

This guide walks through designing a URL shortener — similar to Bit.ly or TinyURL — from the ground up, with a focus on the kind of depth mid-level engineers are expected to bring to the table.

Why This Question Matters for Mid-Level Engineers

Interviewers use the URL shortener problem because it has a small, easy-to-understand core requirement but a huge surface area for follow-up questions. As a mid-level engineer, you're expected to go beyond "it's just a hash map" and demonstrate:

• The ability to estimate scale and translate it into concrete infrastructure decisions
• Familiarity with trade-offs between different encoding and ID-generation strategies
• An understanding of database indexing, sharding, and caching
• Awareness of reliability concerns like collision handling and analytics

Let's build the system step by step.

Step 1: Clarify the Requirements

Before writing any design, nail down functional and non-functional requirements. Interviewers actively evaluate whether you ask good clarifying questions here.

Functional Requirements

• Given a long URL, generate a unique, short alias (e.g., short.ly/aZ9kLm)
• When a user visits the short URL, redirect them to the original long URL
• Support optional custom aliases (e.g., short.ly/my-brand)
• Support link expiration (optional)

Non-Functional Requirements

• High availability: Redirects must work even during partial outages
• Low latency: Redirection should feel instantaneous (ideally under 100ms)
• Scalability: The system should handle a very high read-to-write ratio
• Uniqueness: No two long URLs should collide on the same short code (unless intentional)

Step 2: Estimate Scale (Back-of-the-Envelope Math)

Mid-level candidates are expected to reason about scale quantitatively, not just qualitatively. Assume:

• 100 million new URLs created per month
• Read:write ratio of 100:1 (shorteners are read-heavy — people click links far more often than they create them)

Writes per second: 100,000,000 / (30 × 24 × 3600) ≈ ~40 writes/sec

Reads per second: 40 × 100 = ~4,000 reads/sec

Storage estimate (5 years): 100M URLs/month × 60 months = 6 billion records Each record ~500 bytes (long URL + metadata) → ~3 TB of storage

These numbers immediately tell you two things: this is a read-heavy system that needs aggressive caching, and storage, while large, is manageable with standard sharding — no exotic infrastructure required.

Step 3: High-Level Architecture

A typical URL shortener architecture includes:

1. Client (browser/app) — sends requests to shorten or resolve URLs
2. Load Balancer — distributes traffic across application servers
3. Application Servers — handle URL creation and redirection logic
4. Cache Layer (e.g., Redis) — stores hot short-to-long URL mappings
5. Database — persists the mapping data
6. ID Generation Service — produces unique short codes
7. Analytics Pipeline (optional) — tracks click data asynchronously

Client → Load Balancer → App Servers → Cache → Database
↓
ID Generator Service

Step 4: Designing the Short Code (The Core Problem)

This is where most of the interesting engineering trade-offs live. There are three common approaches.

Option A: Base62 Encoding of an Auto-Incrementing ID

Use a distributed counter or database auto-increment to generate a unique integer, then encode it in Base62 (a-z, A-Z, 0-9).

• A 7-character Base62 string can represent 62^7 ≈ 3.5 trillion unique values — more than enough
• Encoding is deterministic and collision-free by construction
• Downside: A centralized counter can become a bottleneck and a single point of failure at scale

Mitigation: Use a distributed ID generator like Twitter's Snowflake, or pre-allocate ID ranges to each application server so no two servers generate the same ID.

Option B: Hashing the Long URL (MD5/SHA-256 + Truncation)

Hash the original URL and take the first 6–8 characters of the resulting hash.

• Simple and stateless — no coordination needed between servers
• Downside: Collisions are possible (two different URLs producing the same truncated hash) and must be detected and handled, usually by appending a salt and re-hashing

Option C: Pre-generated Key Pool

A background service pre-generates millions of unique random Base62 strings and stores them in a "available keys" table. When a new URL comes in, the application simply pops a key off this pool.

• Removes real-time generation overhead entirely
• Requires a separate service to keep the pool replenished

Recommendation for interviews: Option A (counter + Base62) or Option C (key pool) are generally preferred over hashing, because they avoid collision-handling complexity altogether. Mentioning this trade-off explicitly is a strong signal of mid-level+ thinking.

Step 5: Database Schema

A minimal schema looks like this:

ColumnTypeNotes

short_code

VARCHAR(7)

Primary key, indexed

long_url

TEXT

The original URL

created_at

TIMESTAMP

For expiration/analytics

expires_at

TIMESTAMP

Nullable

user_id

BIGINT

Nullable, for authenticated users

click_count

BIGINT

Denormalized counter, updated async

Database choice: A key-value store (like DynamoDB or Cassandra) fits this access pattern well since lookups are almost always by short_code. If you need relational features (user accounts, custom domains), a sharded relational database (PostgreSQL/MySQL) with short_code as the shard key also works fine.

Step 6: Caching Strategy

Given the 100:1 read-to-write ratio, caching is where most of your system's performance comes from.

• Use Redis or Memcached as a read-through cache in front of the database
• Apply the 80/20 rule: a small fraction of URLs (viral links, marketing campaigns) account for the majority of traffic — these are exactly what benefit from caching
• Use an LRU (Least Recently Used) eviction policy so cold, rarely-accessed links naturally fall out of cache
• On a cache miss, fetch from the database and repopulate the cache (cache-aside pattern)

Step 7: Handling Redirects Correctly

When a user hits short.ly/aZ9kLm, the server needs to decide between:

• HTTP 301 (Permanent Redirect): Browsers cache this aggressively, reducing server load — but you lose the ability to track every single click, since repeat visits may be served from the browser's own cache
• HTTP 302 (Temporary Redirect): Every click hits your server, which is worse for performance but essential if click analytics matter to the business

This is a great trade-off to raise proactively in an interview — it shows you understand that a "simple redirect" has real product implications.

Step 8: Scaling the System

As traffic grows, layer in these scaling techniques:

• Horizontal scaling: Add more stateless application servers behind the load balancer
• Database sharding: Shard by short_code prefix or hash range so no single node becomes a hotspot
• Read replicas: Offload read traffic from the primary database
• CDN edge caching: For extremely popular links, cache redirects at the CDN edge, closer to users
• Rate limiting: Prevent abuse of the URL-creation endpoint with per-user/per-IP limits

Step 9: Additional Considerations Interviewers Like to Probe

• Custom aliases: Requires a uniqueness check against the database before insertion, and a separate reserved-word blocklist
• Expiration and cleanup: A background job (cron or scheduled Lambda) periodically purges expired links from the database and cache
• Analytics: Click events should be written asynchronously (e.g., pushed to a message queue like Kafka) rather than synchronously incrementing a counter on every redirect, to keep the redirect path fast
• Security: Validate and sanitize incoming URLs to prevent the service from being used to redirect to malicious or disallowed domains

Common Mistakes Mid-Level Engineers Make

1. Jumping straight to a database schema without estimating scale first — always frame the numbers before the design
2. Ignoring the read-heavy nature of the system and under-investing in caching
3. Overcomplicating ID generation with hashing when a simple counter-based Base62 approach is cleaner
4. Forgetting redirect semantics (301 vs. 302) and their impact on analytics
5. Not mentioning trade-offs out loud — interviewers care as much about your reasoning process as your final diagram

Final Thoughts

Designing a URL shortener is a compact but complete exercise in distributed systems thinking. It forces you to reason about capacity, choose the right encoding strategy, design an efficient schema, and layer in caching and scaling — all skills that translate directly to real production systems, not just interview whiteboards.

The strongest answers aren't the ones with the fanciest architecture — they're the ones that clearly justify each decision with the numbers and trade-offs behind it. Master that skill here, and it will serve you in almost every other system design question you face.