Coding
Compare implementation quality, correctness, and practical coding ability.
In this genre, the main abilities being tested are Correctness, Completeness, Code Quality.
Unlike system design, this genre focuses more on whether the answer actually works at the code level than on high-level architecture trade-offs.
A high score here does not guarantee strong product judgment, broad architectural thinking, or clear teaching-oriented explanations.
Strong models here are useful for
implementation, debugging, refactoring, and hands-on programming support.
This genre alone cannot tell you
whether the model is best for architecture review, stakeholder writing, or open-ended ideation.
Coding: Claude Fable 5 opens at the top, GPT-5 mini is the most defensible pick
Anthropic
OpenAI
OpenAI
Average score by model
What we weighted
Claude Fable 5 arrived in this genre and immediately took the top spot, winning its opening matchup with the strongest showing on the board. GPT-5.6 has also won everything it has faced so far. Both records are genuinely impressive and genuinely thin: a debut sweep tells you a model can win here, not that it will keep doing so. The exact order at the very top should be read as an early signal.
The most defensible record in the genre belongs to GPT-5 mini: it has faced more coding briefs than any of the leaders and has not lost one. For a light-tier model, an unbeaten run against frontier-class opposition is the standout value story on this table. GPT-5.5, by contrast, posts one of the genre’s best averages but has split its matchups — good code that did not always beat the code across the table. Claude Sonnet 5 landed a solid average in its first outing yet lost it, so its standing undersells its output quality for now.
Correctness dominates the judging here, with completeness and code quality next, so the ranking punishes subtle bugs harder than plain style. The Gemini family has yet to convert a matchup in this genre and sits at the bottom on averages as well. All of this reflects Orivel’s specific tasks and judges: coding covers everything from algorithms to API design, and a handful of briefs cannot cover that space.
Bottom line
GPT-5 mini is the pick you can defend today — unbeaten across the most matchups at light-tier cost. Claude Fable 5 and GPT-5.6 look stronger still, but on early evidence. Watch whether GPT-5.5 starts converting its high-quality answers into wins.
This analysis is derived from Orivel's measured benchmark scores for this genre and is updated periodically. Scores are condition-dependent measurements, not absolute truth.
Top Models in This Genre
This ranking is ordered by average score within this genre only.
Latest Updated: Jul 25, 2026 01:19
Win Rate
Average Score
Win Rate
Average Score
Win Rate
Average Score
Win Rate
Average Score
Win Rate
Average Score
Win Rate
Average Score
Win Rate
Average Score
Win Rate
Average Score
| Ranked Models | Detail | ||||||
|---|---|---|---|---|---|---|---|
| #1 | Claude Fable 5 | Anthropic |
100%
|
91
|
1 | 1 | View scores and evaluation for Claude Fable 5 |
| #2 | GPT-5.6 | OpenAI |
100%
|
86
|
2 | 2 | View scores and evaluation for GPT-5.6 |
| #3 | GPT-5 mini | OpenAI |
100%
|
82
|
5 | 5 | View scores and evaluation for GPT-5 mini |
| #4 | GPT-5.5 | OpenAI |
50%
|
89
|
1 | 2 | View scores and evaluation for GPT-5.5 |
| #5 | Claude Sonnet 5 NEW | Anthropic |
0%
|
83
|
0 | 1 | View scores and evaluation for Claude Sonnet 5 |
| #6 | Gemini 2.5 Pro |
0%
|
74
|
0 | 5 | View scores and evaluation for Gemini 2.5 Pro | |
| #7 | Gemini 2.5 Flash-Lite |
0%
|
72
|
0 | 3 | View scores and evaluation for Gemini 2.5 Flash-Lite | |
| #8 | Gemini 2.5 Flash |
0%
|
68
|
0 | 5 | View scores and evaluation for Gemini 2.5 Flash |
What Is Evaluated in Coding
Scoring criteria and weight used for this genre ranking.
Correctness
35.0%
This criterion is included to check Correctness in the answer. It carries heavier weight because this part strongly shapes the overall result in this genre.
Completeness
20.0%
This criterion is included to check Completeness in the answer. It has meaningful weight because it affects quality in a visible way, even if it is not the only thing that matters.
Code Quality
20.0%
This criterion is included to check Code Quality in the answer. It has meaningful weight because it affects quality in a visible way, even if it is not the only thing that matters.
Practical Value
15.0%
This criterion is included to check Practical Value in the answer. It is weighted more lightly because it supports the main goal rather than defining the genre by itself.
Instruction Following
10.0%
This criterion is included to check Instruction Following in the answer. It is weighted more lightly because it supports the main goal rather than defining the genre by itself.
Recent tasks
Coding
Web Server Log Analyzer
Write a Python function analyze_logs(log_data) that takes a multi-line string containing web server log entries. The function should parse these logs, perform an analysis, and return a dictionary summarizing the results. Each valid log line follows this format: [TIMESTAMP] LEVEL IP_ADDRESS "REQUEST_METHOD /path" RESPONSE_CODE BYTES_SENT Example of a valid line: [2023-10-27T10:00:00Z] INFO 192.168.1.1 "GET /index.html" 200 1543 Your function should: Parse only the valid log lines, gracefully ignoring any malformed or empty lines. Calculate the following metrics: total_requests: The total count of valid log entries. error_rate: The percentage of requests with a LEVEL of ERROR, rounded to two decimal places. top_3_ips: A list of tuples, where each tuple contains an IP address and its request count, for the top 3 most frequent IPs. The list should be sorted in descending order of request count. busiest_hour: The hour of the day (an integer from 0 to 23) that had the most requests. The timestamp is in ISO 8601 format (UTC). Return a dictionary with keys total_requests, error_rate, top_3_ips, and busiest_hour containing the calculated values. Handle the following edge cases: If the input string log_data is empty, return a dictionary with zeroed or empty values as appropriate (e.g., total_requests: 0, top_3_ips: []). If there are fewer than 3 unique IP addresses, the top_3_ips list should contain all unique IPs, sorted by count. If there is a tie for the busiest hour, returning any one of the tied hours is acceptable.
Coding
Rate Limiter with Sliding Window and Fair Multi-Tenant Quotas
Implement a reusable rate limiter library in a language of your choice (Python, Go, TypeScript, Java, or Rust) that enforces per-client request quotas using a sliding-window algorithm, plus a fair-sharing policy across multiple tenants. Functional requirements: Provide a class or module with a method such as allow(tenant_id, client_id, now_ms) that returns whether a request is permitted and, when denied, how many milliseconds until the next request would be allowed (retry_after_ms). Each client is limited to a maximum number of requests within a rolling time window (for example, 100 requests per 60,000 ms). Configuration must be adjustable per tenant. Implement a true sliding window (weighted or log-based), not a fixed calendar-bucket window, so that bursts across bucket boundaries are handled correctly. Add a per-tenant global cap so that all clients of a tenant combined cannot exceed a tenant-level ceiling, and when the tenant is saturated, remaining capacity is shared fairly across active clients rather than being monopolized by one client. The limiter must be safe under concurrent access from multiple threads or async tasks. Memory must not grow unbounded: stale client state must be evicted or compacted over time. Deliverables: The complete implementation with clear public API and inline documentation of key decisions. A brief explanation (in comments or a short prose section) of the sliding-window algorithm you chose and its accuracy/memory tradeoffs. A test suite covering the core edge cases described below. Edge cases to address explicitly in code and tests: Requests exactly at the window boundary. A client that goes idle then returns after the window has fully elapsed. Concurrent requests racing against the same client counter. Clock going backwards or duplicate timestamps. Tenant saturation and fair redistribution among competing clients. Eviction of stale client state without dropping active clients. State any assumptions you make (single-process vs distributed, monotonic clock availability, etc.). If you assume a single process, briefly describe how the design would extend to a distributed deployment.
Coding
Implement a Deterministic Limit Order Book Simulator
Write a single-file Python 3.11 solution implementing the function process_events(events: list[dict]) -> dict. Do not use external packages. The function must simulate a small exchange limit order book for one instrument. It receives a list of event dictionaries in input order and returns a dictionary with exactly these keys: trades, rejected, book. Event types: New order event: Required fields: type="new", id, side, order_type, qty. side is "buy" or "sell". order_type is "limit" or "market". qty is a positive integer. A limit order also requires price, a positive integer number of cents. Optional field tif is time-in-force: "GTC", "IOC", or "FOK". If absent, use "GTC" for limit orders and "IOC" for market orders. Market orders may not have tif="GTC" and may not rest on the book. Cancel event: Required fields: type="cancel", id. It cancels the remaining quantity of a currently resting order with that id. Matching rules: The book has bids and asks. Resting buy limit orders are bids; resting sell limit orders are asks. Price-time priority is mandatory: best price first; for the same price, earlier accepted resting order first. A buy order matches resting asks while it can cross: market buy crosses any ask; limit buy crosses asks with ask price <= buy limit price. A sell order matches resting bids while it can cross: market sell crosses any bid; limit sell crosses bids with bid price >= sell limit price. Each trade quantity is min(incoming remaining quantity, resting remaining quantity). Trade price is always the resting maker order's limit price, never the incoming order's price. A trade record must be appended immediately when it happens with exactly these keys: buy_id, sell_id, price, qty, taker_id, maker_id. Partially filled resting orders keep their original priority with the remaining quantity. Fully filled orders leave the book. Time-in-force behavior: GTC limit orders rest any unfilled remainder on the book. IOC orders execute as much as possible immediately, then cancel any remainder. FOK orders must be completely fillable immediately according to the current book and crossing rules. If not completely fillable, they produce no trades and do not change the book. If completely fillable, execute normally. FOK orders never rest. Validation and rejection rules: If an event is malformed, reject it without changing the book. Append a rejection record to rejected with keys input_index, event, reason. The reason may be a short human-readable string. Reject a new order if its id is already used by any previously accepted new order, even if that earlier order has since filled or been canceled. Reject cancel events for unknown ids or ids that are no longer resting. Reject non-integer, zero, or negative qty and price values. In Python, bool must not be accepted as an integer for these fields. Ignore extra fields on otherwise valid events. Return format: trades: list of trade records in execution order. rejected: list of rejection records in input order. book: a dictionary with keys bids and asks. book["bids"] must list all resting bids sorted by descending price, then original resting time, each as {"id": id, "price": price, "qty": remaining_qty}. book["asks"] must list all resting asks sorted by ascending price, then original resting time, each as {"id": id, "price": price, "qty": remaining_qty}. Your answer should be complete executable Python code defining process_events. You may include helper classes/functions and a small self-test section guarded by if name == "main":, but the core function must not read from stdin or write to stdout.
Coding
Implement Atomic JSON Patch Application in Python
Write a Python 3.11 implementation of a function named apply_json_patch(document, patch) that applies a JSON Patch-style sequence of operations to a JSON-compatible value and returns the patched value. The input document may be any combination of dict, list, str, int, float, bool, and None. The patch is a list of operation dicts. The implementation must not mutate the original document or any nested object reachable from it. If any operation is invalid, the function must raise a custom exception class named JsonPatchError and leave the original document unchanged. Supported operations are add, remove, replace, move, copy, and test. Use JSON Pointer paths with slash-separated tokens, where the empty string identifies the whole document, tokens decode ~1 as / and ~0 as ~, and any other use of ~ is invalid. For objects, a path token is a key. For arrays, a path token must be a non-negative integer without leading zeros except the single token 0; for add only, the final token may be - to append. The add operation inserts into arrays at an index from 0 through len(array), appends for -, sets an object key, or replaces the whole document at path empty. The remove operation requires the target to exist and deletes it. The replace operation requires the target to exist and replaces it. The move operation requires from and path, removes the value at from and adds it at path, and must reject moving a value into one of its own descendants. The copy operation requires from and path and deep-copies the source value to the target. The test operation requires value and succeeds only if the current target is deeply equal to value, including normal Python equality for numbers and exact equality for strings, booleans, and None. Each operation dict must contain exactly the fields required for that operation plus the op field; unknown fields or missing fields are errors. The function should be deterministic, reasonably efficient, and rely only on the Python standard library. Include any helper functions or classes needed. Do not write a command-line program or use external packages.
Coding
Implement a Dependency-Based Task Scheduler in Python
Write a Python function or class that schedules a list of tasks based on their dependencies. The scheduler should determine the order in which tasks can be executed, grouping tasks that can run in parallel. The input will be a list of dictionaries, where each dictionary represents a task with the following keys: id: A unique string identifier for the task. name: A string name for the task. dependencies: A list of string IDs of tasks that must be completed before this task can start. Your implementation should: Take the list of task dictionaries as input. Return a valid execution plan as a list of lists. Each inner list represents a 'batch' of tasks that can be executed concurrently. The order of batches represents the sequential execution order. The order of task IDs within a batch does not matter. Detect and handle circular dependencies. If a cycle is found, it should raise a ValueError with a descriptive message. Detect and handle cases where a dependency ID does not correspond to any existing task. This should also raise a ValueError.
Coding
Rate Limiter with Sliding Window and Burst Allowance
Design and implement a thread-safe rate limiter in a language of your choice (Python, Go, Java, TypeScript, or Rust) that supports the following requirements: API surface: Expose at least these operations: allow(client_id: str, cost: int = 1) -> bool — returns whether the request is permitted right now. retry_after(client_id: str) -> float — returns seconds until at least 1 unit of capacity is available (0 if currently allowed). A constructor that accepts per-client configuration: rate (units per second), burst (max units stored), and an optional window_seconds for sliding-window accounting. Algorithm: Implement a hybrid that combines a token bucket (for burst tolerance) with a sliding-window log or counter (to bound the total requests permitted within window_seconds, preventing sustained abuse that a pure token bucket would allow after refills). A request is permitted only if both checks pass. Justify your data-structure choice for the sliding window (exact log vs. weighted two-bucket approximation) and discuss memory/accuracy tradeoffs in a short comment block or accompanying note. Concurrency: The limiter will be hit by many threads/goroutines concurrently for the same and different client_ids. Avoid a single global lock becoming a bottleneck (e.g., per-client locks or lock striping). Document why your approach is correct under concurrent allow calls (no double-spend of tokens, no lost updates). Time source: Make the clock injectable so tests are deterministic. Use a monotonic clock by default. Edge cases to handle explicitly: cost larger than burst (must reject, never block forever). Clock going backwards or large pauses (e.g., suspended VM): clamp rather than crash, and don't grant unbounded tokens. First-ever request for a new client (lazy initialization). Stale client cleanup (memory must not grow unbounded if clients stop calling). Fractional tokens / sub-millisecond timing. Tests: Provide at least 6 unit tests using the injectable clock that cover: basic allow/deny, burst draining and refill, sliding-window cap independent of bucket refill, cost > burst, concurrent contention on one client (deterministic property: total permitted in T seconds ≤ rate*T + burst), and stale-client eviction. Complexity: State the amortized time complexity of allow and the memory complexity per client. Deliver: complete runnable code (single file is fine, but you may split files if you label them clearly), the tests, and a brief design note (max ~250 words) explaining your choices and the precise semantics when the two algorithms disagree.