Orivel Orivel
Open menu

In-Memory Key-Value Store with Transaction Support

Compare model answers for this Coding benchmark and review scores, judging comments, and related examples.

Login or register to use likes and favorites. Register

X f L

Contents

Task Overview

Benchmark Genres

Coding

Task Creator Model

Answering Models

Judge Models

Task Prompt

Write a Python class `InMemoryDB` that implements a simple in-memory key-value data store with support for nested transactions. The class should have the following methods: - `get(key)`: Returns the value associated with a key. If the key does not exist, it should return `None`. - `set(key, value)`: Sets the value for a given key. If a transaction is in progress, this change should only be visible within that transaction until it is committed. - `begin()`: Starts a new transaction. Transactions can be nested. - `c...

Show more

Write a Python class `InMemoryDB` that implements a simple in-memory key-value data store with support for nested transactions. The class should have the following methods: - `get(key)`: Returns the value associated with a key. If the key does not exist, it should return `None`. - `set(key, value)`: Sets the value for a given key. If a transaction is in progress, this change should only be visible within that transaction until it is committed. - `begin()`: Starts a new transaction. Transactions can be nested. - `commit()`: Commits all changes made in the current transaction to its parent transaction (or to the main store if it's the outermost transaction). If there is no active transaction, it should raise an error. - `rollback()`: Discards all changes made in the current transaction. If there is no active transaction, it should raise an error.

Task Context

The key challenge is managing state across nested transactions. A `rollback` should only undo changes made in the most recent, uncommitted transaction. A `commit` should merge the changes from the current transaction into the parent transaction's scope. Only when the outermost transaction is committed do the changes become permanent in the main data store. Example usage: ```python db = InMemoryDB() # No transaction print(db.get("a")) # Expected: None db.set("a", 10) print(db.get("a")) # Expected: 10 # Transact...

Show more

The key challenge is managing state across nested transactions. A `rollback` should only undo changes made in the most recent, uncommitted transaction. A `commit` should merge the changes from the current transaction into the parent transaction's scope. Only when the outermost transaction is committed do the changes become permanent in the main data store. Example usage: ```python db = InMemoryDB() # No transaction print(db.get("a")) # Expected: None db.set("a", 10) print(db.get("a")) # Expected: 10 # Transaction db.begin() db.set("a", 20) print(db.get("a")) # Expected: 20 # Nested transaction db.begin() db.set("a", 30) print(db.get("a")) # Expected: 30 # Rollback nested db.rollback() print(db.get("a")) # Expected: 20 # Commit outer db.commit() print(db.get("a")) # Expected: 20 # Error cases try: db.commit() # No transaction active except Exception as e: print(f"Error: {e}") ```

Judging Policy

A high-quality solution will correctly implement all specified methods. The core of the evaluation will focus on the correct handling of nested transactions. - **Correctness:** The `get`, `set`, `begin`, `commit`, and `rollback` operations must work as described, especially in scenarios with multiple nested transactions. The state of the data store must be consistent after each operation. A `get` operation should correctly resolve the value by checking the current transaction first, then its parent, and so on, up t...

Show more

A high-quality solution will correctly implement all specified methods. The core of the evaluation will focus on the correct handling of nested transactions. - **Correctness:** The `get`, `set`, `begin`, `commit`, and `rollback` operations must work as described, especially in scenarios with multiple nested transactions. The state of the data store must be consistent after each operation. A `get` operation should correctly resolve the value by checking the current transaction first, then its parent, and so on, up to the main store. - **Robustness:** The code should handle edge cases gracefully, such as attempting to `commit` or `rollback` when no transaction is active (by raising an appropriate error, like `ValueError` or a custom exception). It should also correctly handle setting a key multiple times within the same or different transaction levels. - **Design and Readability:** The internal data structures used to manage the main store and the transaction logs should be logical and efficient. The code should be well-structured, clean, and easy to understand. Using a stack-based approach for managing transaction states is a common and effective pattern.

Answers

class InMemoryDB: def __init__(self): self._main_store = {} self._transaction_stack = [] def get(self, key): # Search from the most recent transaction backwards for txn in reversed(self._transaction_stack): if key in txn: return txn[key] return self._main_store.get(key, None) def set(self, key, value): if self._transaction_stack: self._transaction_stack[-1][key] = value else: self._main_store[key] = value def begin(self): self._transaction_stack.append({}) def commit(self):...

Show Full Answer

class InMemoryDB: def __init__(self): self._main_store = {} self._transaction_stack = [] def get(self, key): # Search from the most recent transaction backwards for txn in reversed(self._transaction_stack): if key in txn: return txn[key] return self._main_store.get(key, None) def set(self, key, value): if self._transaction_stack: self._transaction_stack[-1][key] = value else: self._main_store[key] = value def begin(self): self._transaction_stack.append({}) def commit(self): if not self._transaction_stack: raise Exception("No active transaction to commit") changes = self._transaction_stack.pop() if self._transaction_stack: # Merge into parent transaction self._transaction_stack[-1].update(changes) else: # Merge into main store self._main_store.update(changes) def rollback(self): if not self._transaction_stack: raise Exception("No active transaction to rollback") self._transaction_stack.pop() # Example usage if __name__ == "__main__": db = InMemoryDB() # No transaction print(db.get("a")) # Expected: None db.set("a", 10) print(db.get("a")) # Expected: 10 # Transaction db.begin() db.set("a", 20) print(db.get("a")) # Expected: 20 # Nested transaction db.begin() db.set("a", 30) print(db.get("a")) # Expected: 30 # Rollback nested db.rollback() print(db.get("a")) # Expected: 20 # Commit outer db.commit() print(db.get("a")) # Expected: 20 # Error cases try: db.commit() # No transaction active except Exception as e: print(f"Error: {e}")

Result

#2

Winning Votes

0 / 3

Average Score

84

Total Score

83

Overall Comments

Answer A is a correct and well-structured implementation of the InMemoryDB class. It uses a stack-based approach with clear variable naming (_main_store, _transaction_stack). The logic for get, set, begin, commit, and rollback is all correct and handles nested transactions properly. It raises a generic Exception for error cases, which is functional but less semantically precise than using RuntimeError or ValueError. The example usage is included and matches the expected output. Overall a solid solution with minor stylistic shortcomings.

View Score Details

Correctness

Weight 35%
90

Answer A correctly implements all methods. The get method searches the transaction stack in reverse order before falling back to the main store. The set method writes to the current transaction or main store as appropriate. Commit merges changes upward correctly. Rollback discards the current transaction. All nested transaction scenarios are handled correctly.

Completeness

Weight 20%
85

Answer A implements all five required methods (get, set, begin, commit, rollback) and includes the full example usage from the task prompt. Error cases are handled. No missing functionality.

Code Quality

Weight 20%
75

Answer A uses clear variable names and a logical structure. However, it raises a generic Exception for error cases rather than a more specific exception type like RuntimeError or ValueError, which is a minor code quality issue. The code is otherwise clean and readable.

Practical Value

Weight 15%
75

Answer A is practically useful as a reference implementation. The example usage demonstrates all key scenarios. The generic Exception is a minor practical concern since callers may want to catch specific exception types.

Instruction Following

Weight 10%
85

Answer A follows all instructions: implements the required class and all five methods, handles nested transactions, raises errors for commit/rollback without active transactions, and includes the example usage from the prompt.

Judge Models OpenAI GPT-5.2

Total Score

74

Overall Comments

Implements the required API with a clear transaction stack and correct nested begin/commit/rollback behavior for simple set/get semantics. The logic of resolving keys from the latest transaction to the main store is sound. Minor issues are mainly around robustness and polish: it raises a generic Exception type, and there is no explicit guidance or handling for deletion semantics (not required) or for distinguishing an explicitly set None from missing (not required but sometimes relevant).

View Score Details

Correctness

Weight 35%
77

Correctly supports nested begin/commit/rollback via a stack of dicts; get searches from inner to outer; commit merges into parent or main store; rollback discards current changes. No functional errors for the specified operations.

Completeness

Weight 20%
75

Implements all required methods and includes example usage demonstrating nested transactions and error case. Does not address optional behaviors like deletes (not requested).

Code Quality

Weight 20%
70

Readable and straightforward, with clear member names; slight drawback is generic Exception usage and minor extra verbosity (explicit None default in get).

Practical Value

Weight 15%
68

Practical for basic transactional KV usage; lacks extras like custom exception types, deletion markers, or transaction introspection, but those are beyond the prompt.

Instruction Following

Weight 10%
80

Follows the requested interface and behavior, including nested transactions and raising an error when committing/rolling back without an active transaction.

Total Score

93

Overall Comments

Answer A provides a fully correct and functional implementation of the in-memory database. The logic for handling nested transactions is sound, and the code is clean and readable, with particularly descriptive variable names like `_transaction_stack`. Its main weakness is the use of the generic `Exception` for error handling, which is generally discouraged in favor of more specific exception types.

View Score Details

Correctness

Weight 35%
100

The implementation is fully correct. The logic for `get`, `set`, `commit`, and `rollback` correctly handles nested transactions by searching through the transaction stack in the correct order and merging/discarding changes appropriately.

Completeness

Weight 20%
100

The answer is complete, implementing all five required methods: `get`, `set`, `begin`, `commit`, and `rollback`.

Code Quality

Weight 20%
85

The code is clean and readable. The variable names, especially `_transaction_stack`, are very descriptive. The only minor drawback is the use of the generic `Exception` class for error handling.

Practical Value

Weight 15%
75

The solution provides a good, clear implementation of a common data structure pattern. It's a valuable educational example, though it lacks features like concurrency control for production use.

Instruction Following

Weight 10%
100

The answer perfectly follows all instructions. It implements the specified class and methods, handles the logic as described, and includes the example usage block.

class InMemoryDB: def __init__(self): self._store = {} self._transactions = [] def get(self, key): for tx in reversed(self._transactions): if key in tx: return tx[key] return self._store.get(key) def set(self, key, value): if self._transactions: self._transactions[-1][key] = value else: self._store[key] = value def begin(self): self._transactions.append({}) def commit(self): if not self._transactions: raise RuntimeError("No active transaction to commit")...

Show Full Answer

class InMemoryDB: def __init__(self): self._store = {} self._transactions = [] def get(self, key): for tx in reversed(self._transactions): if key in tx: return tx[key] return self._store.get(key) def set(self, key, value): if self._transactions: self._transactions[-1][key] = value else: self._store[key] = value def begin(self): self._transactions.append({}) def commit(self): if not self._transactions: raise RuntimeError("No active transaction to commit") current = self._transactions.pop() if self._transactions: self._transactions[-1].update(current) else: self._store.update(current) def rollback(self): if not self._transactions: raise RuntimeError("No active transaction to rollback") self._transactions.pop() if __name__ == "__main__": db = InMemoryDB() # No transaction print(db.get("a")) # Expected: None db.set("a", 10) print(db.get("a")) # Expected: 10 # Transaction db.begin() db.set("a", 20) print(db.get("a")) # Expected: 20 # Nested transaction db.begin() db.set("a", 30) print(db.get("a")) # Expected: 30 # Rollback nested db.rollback() print(db.get("a")) # Expected: 20 # Commit outer db.commit() print(db.get("a")) # Expected: 20 # Error cases try: db.commit() # No transaction active except Exception as e: print(f"Error: {e}")

Result

#1 | Winner

Winning Votes

3 / 3

Average Score

85

Total Score

84

Overall Comments

Answer B is functionally equivalent to Answer A and also correctly implements all required methods. It uses RuntimeError instead of the generic Exception for error handling, which is more semantically appropriate and Pythonic. The variable names (_store, _transactions) are slightly more concise. The logic is identical in correctness and the example usage is the same. The use of RuntimeError is a meaningful improvement in robustness and code quality over Answer A's generic Exception.

View Score Details

Correctness

Weight 35%
90

Answer B is functionally identical to Answer A in terms of correctness. All methods work correctly for nested transactions, commit, rollback, and error cases. No correctness issues found.

Completeness

Weight 20%
85

Answer B also implements all five required methods and includes the full example usage. Error cases are handled. Completeness is on par with Answer A.

Code Quality

Weight 20%
80

Answer B uses RuntimeError for error cases, which is more semantically appropriate and Pythonic than a generic Exception. Variable names are concise and the code is clean. This gives it a slight edge in code quality over Answer A.

Practical Value

Weight 15%
75

Answer B is equally practical. Using RuntimeError makes it slightly easier for callers to handle errors specifically, which is a minor practical advantage.

Instruction Following

Weight 10%
85

Answer B also follows all instructions completely. All required methods are implemented, nested transactions are handled, and errors are raised appropriately. Instruction following is equivalent to Answer A.

Judge Models OpenAI GPT-5.2

Total Score

75

Overall Comments

Implements the same correct transaction-stack approach with clean, readable code and proper nested transaction behavior. It improves robustness slightly by raising a more specific RuntimeError for invalid commit/rollback calls. Overall it matches the prompt well and is a bit more idiomatic and precise in error signaling, while otherwise maintaining comparable clarity and correctness.

View Score Details

Correctness

Weight 35%
78

Same correct stack-based semantics as A; nested visibility and rollback/commit behavior are correct for the specified set/get operations.

Completeness

Weight 20%
75

Implements all required methods and includes the same example usage coverage; meets the prompt requirements.

Code Quality

Weight 20%
72

Clean and idiomatic, consistent naming, concise methods; uses a more appropriate exception type, slightly improving overall quality.

Practical Value

Weight 15%
69

Similarly practical baseline transactional KV store; marginally better for real use due to clearer error signaling via RuntimeError.

Instruction Following

Weight 10%
81

Also follows the interface and behavior precisely; error raising is explicit and appropriately typed, aligning well with the prompt’s guidance.

Total Score

94

Overall Comments

Answer B also delivers a correct and well-designed solution that is functionally identical to Answer A. It correctly implements all features, including nested transactions. The code is clear and easy to follow. Its key strength over Answer A is its use of a more appropriate, specific exception (`RuntimeError`) when handling invalid operations, which demonstrates better coding practices for robustness.

View Score Details

Correctness

Weight 35%
100

The implementation is fully correct. The logic for handling nested transactions is sound and identical to Answer A's. All operations behave as expected according to the prompt's examples.

Completeness

Weight 20%
100

The answer is complete. It provides a full implementation of the `InMemoryDB` class with all the methods specified in the prompt.

Code Quality

Weight 20%
90

The code quality is very high. The logic is clean and efficient. It makes a better choice of exception (`RuntimeError`) than Answer A, which is more specific and robust. Variable names are clear, though slightly less descriptive than in A.

Practical Value

Weight 15%
75

This is a solid and practical implementation for its intended purpose as a simple, single-threaded in-memory store. It serves as an excellent example of managing state with a stack.

Instruction Following

Weight 10%
100

The answer adheres to all instructions in the prompt. The class methods, behavior, and error handling match the requirements precisely. The example usage is also included and works correctly.

Comparison Summary

Final rank order is determined by judge-wise rank aggregation (average rank + Borda tie-break). Average score is shown for reference.

Judges: 3

Winning Votes

0 / 3

Average Score

84
View this answer

Winning Votes

3 / 3

Average Score

85
View this answer

Judging Results

Why This Side Won

Both answers provide nearly identical, correct, and well-structured solutions. However, Answer B is slightly superior due to its choice of a more specific exception (`RuntimeError`) for error conditions, which is better practice than Answer A's use of the generic `Exception`. This demonstrates a better understanding of robust error handling. While Answer A has slightly more descriptive variable names, the improvement in exception handling makes Answer B the better overall implementation.

Judge Models OpenAI GPT-5.2

Why This Side Won

Both solutions correctly handle nested transactions using a stack and resolve reads from the most recent transaction outward. Answer B wins because it is equally correct and complete but is slightly more robust and idiomatic by using a more specific exception type (RuntimeError) instead of a generic Exception, with similarly clean structure.

Why This Side Won

Both answers are functionally equivalent and correct. Answer B edges out Answer A primarily due to its use of RuntimeError for error cases, which is more semantically appropriate and Pythonic than the generic Exception used in Answer A. This makes Answer B slightly more robust and better in code quality, which are important criteria in this benchmark. All other aspects are essentially identical.

X f L