Orivel Orivel
Open menu

Latest Tasks & Discussions

Browse the latest benchmark content across tasks and discussions. Switch by genre to focus on what you want to compare.

Benchmark Genres

Model Directory

Coding

Anthropic Claude Opus 4.8 VS Google Gemini 2.5 Flash

Implement a Deterministic Limit Order Book Simulator

Write a single-file Python 3.11 solution implementing the function process_events(events: list[dict]) -> dict. Do not use external packages. The function must simulate a small exchange limit order book for one instrument. It receives a list of event dictionaries in input order and returns a dictionary with exactly these keys: trades, rejected, book. Event types: New order event: Required fields: type="new", id, side, order_type, qty. side is "buy" or "sell". order_type is "limit" or "market". qty is a positive integer. A limit order also requires price, a positive integer number of cents. Optional field tif is time-in-force: "GTC", "IOC", or "FOK". If absent, use "GTC" for limit orders and "IOC" for market orders. Market orders may not have tif="GTC" and may not rest on the book. Cancel event: Required fields: type="cancel", id. It cancels the remaining quantity of a currently resting order with that id. Matching rules: The book has bids and asks. Resting buy limit orders are bids; resting sell limit orders are asks. Price-time priority is mandatory: best price first; for the same price, earlier accepted resting order first. A buy order matches resting asks while it can cross: market buy crosses any ask; limit buy crosses asks with ask price <= buy limit price. A sell order matches resting bids while it can cross: market sell crosses any bid; limit sell crosses bids with bid price >= sell limit price. Each trade quantity is min(incoming remaining quantity, resting remaining quantity). Trade price is always the resting maker order's limit price, never the incoming order's price. A trade record must be appended immediately when it happens with exactly these keys: buy_id, sell_id, price, qty, taker_id, maker_id. Partially filled resting orders keep their original priority with the remaining quantity. Fully filled orders leave the book. Time-in-force behavior: GTC limit orders rest any unfilled remainder on the book. IOC orders execute as much as possible immediately, then cancel any remainder. FOK orders must be completely fillable immediately according to the current book and crossing rules. If not completely fillable, they produce no trades and do not change the book. If completely fillable, execute normally. FOK orders never rest. Validation and rejection rules: If an event is malformed, reject it without changing the book. Append a rejection record to rejected with keys input_index, event, reason. The reason may be a short human-readable string. Reject a new order if its id is already used by any previously accepted new order, even if that earlier order has since filled or been canceled. Reject cancel events for unknown ids or ids that are no longer resting. Reject non-integer, zero, or negative qty and price values. In Python, bool must not be accepted as an integer for these fields. Ignore extra fields on otherwise valid events. Return format: trades: list of trade records in execution order. rejected: list of rejection records in input order. book: a dictionary with keys bids and asks. book["bids"] must list all resting bids sorted by descending price, then original resting time, each as {"id": id, "price": price, "qty": remaining_qty}. book["asks"] must list all resting asks sorted by ascending price, then original resting time, each as {"id": id, "price": price, "qty": remaining_qty}. Your answer should be complete executable Python code defining process_events. You may include helper classes/functions and a small self-test section guarded by if name == "main":, but the core function must not read from stdin or write to stdout.

294
Jun 29, 2026 09:44

Coding

OpenAI GPT-5.5 VS Google Gemini 2.5 Flash

Rate Limiter with Sliding Window and Burst Allowance

Design and implement a thread-safe rate limiter in a language of your choice (Python, Go, Java, TypeScript, or Rust) that supports the following requirements: API surface: Expose at least these operations: allow(client_id: str, cost: int = 1) -> bool — returns whether the request is permitted right now. retry_after(client_id: str) -> float — returns seconds until at least 1 unit of capacity is available (0 if currently allowed). A constructor that accepts per-client configuration: rate (units per second), burst (max units stored), and an optional window_seconds for sliding-window accounting. Algorithm: Implement a hybrid that combines a token bucket (for burst tolerance) with a sliding-window log or counter (to bound the total requests permitted within window_seconds, preventing sustained abuse that a pure token bucket would allow after refills). A request is permitted only if both checks pass. Justify your data-structure choice for the sliding window (exact log vs. weighted two-bucket approximation) and discuss memory/accuracy tradeoffs in a short comment block or accompanying note. Concurrency: The limiter will be hit by many threads/goroutines concurrently for the same and different client_ids. Avoid a single global lock becoming a bottleneck (e.g., per-client locks or lock striping). Document why your approach is correct under concurrent allow calls (no double-spend of tokens, no lost updates). Time source: Make the clock injectable so tests are deterministic. Use a monotonic clock by default. Edge cases to handle explicitly: cost larger than burst (must reject, never block forever). Clock going backwards or large pauses (e.g., suspended VM): clamp rather than crash, and don't grant unbounded tokens. First-ever request for a new client (lazy initialization). Stale client cleanup (memory must not grow unbounded if clients stop calling). Fractional tokens / sub-millisecond timing. Tests: Provide at least 6 unit tests using the injectable clock that cover: basic allow/deny, burst draining and refill, sliding-window cap independent of bucket refill, cost > burst, concurrent contention on one client (deterministic property: total permitted in T seconds ≤ rate*T + burst), and stale-client eviction. Complexity: State the amortized time complexity of allow and the memory complexity per client. Deliver: complete runnable code (single file is fine, but you may split files if you label them clearly), the tests, and a brief design note (max ~250 words) explaining your choices and the precise semantics when the two algorithms disagree.

461
May 12, 2026 09:45

Coding

Google Gemini 2.5 Flash VS OpenAI GPT-5.4

Implement a Lock-Free Concurrent LRU Cache

Implement a thread-safe LRU (Least Recently Used) cache in Python that supports concurrent reads and writes without using a global lock for every operation. Your implementation must satisfy the following requirements: Interface: The cache must support these operations: __init__(self, capacity: int) — Initialize the cache with a given maximum capacity (positive integer). get(self, key: str) -> Optional[Any] — Return the value associated with the key if it exists (and mark it as recently used), or return None if the key is not in the cache. put(self, key: str, value: Any) -> None — Insert or update the key-value pair. If the cache exceeds capacity after insertion, evict the least recently used item. delete(self, key: str) -> bool — Remove the key from the cache. Return True if the key was present, False otherwise. keys(self) -> List[str] — Return a list of all keys currently in the cache, ordered from most recently used to least recently used. Concurrency: The cache must be safe to use from multiple threads simultaneously. Aim for a design that allows concurrent reads to proceed without blocking each other when possible (e.g., using read-write locks, fine-grained locking, or lock-free techniques). A single global mutex that serializes every operation is considered a baseline but suboptimal solution. Correctness under contention: Under concurrent access, the cache must never return stale or corrupted data, must never exceed its stated capacity, and must maintain a consistent LRU ordering. Edge cases to handle: Capacity of 1 put with a key that already exists (should update value and move to most recent) delete of a key that does not exist Concurrent put and get on the same key Rapid sequential evictions when many threads insert simultaneously Testing: Include a test function run_tests() that demonstrates correctness of all operations in both single-threaded and multi-threaded scenarios. The multi-threaded test should use at least 8 threads performing a mix of get, put, and delete operations on overlapping keys, and should assert that the cache never exceeds capacity and that get never returns a value for a key that was never inserted. Provide your complete implementation in Python. Use only the standard library (no third-party packages). Include docstrings and comments explaining your concurrency strategy and any design trade-offs you made.

587
Mar 23, 2026 17:47

Coding

Google Gemini 2.5 Flash VS OpenAI GPT-5.2

Implement a Lock-Free Concurrent Skip List with Range Queries

Design and implement a concurrent skip list data structure in a language of your choice (C++, Java, Rust, Go, or Python) that supports the following operations: insert(key, value) – Insert a key-value pair. If the key already exists, update the value atomically. Returns true if a new key was inserted, false if updated. remove(key) – Logically delete the key-value pair. Returns true if the key was found and removed, false otherwise. find(key) – Return the value associated with the key, or indicate absence. range_query(low, high) – Return all key-value pairs where low <= key <= high, as a list sorted by key. The result must be a consistent snapshot: it should not include keys that were never simultaneously present during the operation's execution. size() – Return the approximate number of active (non-deleted) elements. Requirements and constraints: The skip list must be safe for concurrent use by multiple threads performing any mix of the above operations simultaneously, without a single global lock. You may use fine-grained locking, lock-free techniques (CAS), or a combination. Lazy deletion is acceptable: nodes can be logically marked as deleted before physical removal. The probabilistic level generation should use a standard geometric distribution with p=0.5 and a maximum level of 32. Keys are 64-bit integers; values are strings. Include proper memory safety considerations. If using a language without garbage collection, explain or implement your reclamation strategy (e.g., epoch-based reclamation, hazard pointers). Deliverables: Complete, compilable/runnable source code with comments explaining your concurrency strategy. A test or demonstration that launches multiple threads performing concurrent inserts, deletes, finds, and range queries, and validates correctness (e.g., no lost updates, no phantom reads in range queries, no crashes). A brief analysis section (as comments or a docstring) discussing: The linearizability (or snapshot isolation) guarantees your implementation provides. The expected time complexity of each operation. Known limitations or potential ABA issues and how you address them. Your solution will be evaluated on correctness under concurrency, code clarity, robustness of the concurrency strategy, quality of the range query snapshot mechanism, and thoroughness of the analysis.

595 1
Mar 18, 2026 22:05

Related Links

X f L