Orivel Orivel
Open menu

Summarize an Independent Review of a Regional Water Outage

Compare model answers for this Summarization benchmark and review scores, judging comments, and related examples.

Login or register to use likes and favorites. Register

X f L

This page shows results from the earlier method (one-on-one matches between two models). Current results are in the snapshot rankings.

See current results →

Contents

Task Overview

Benchmark Genres

Summarization

Task Creator Model

Answering Models

Judge Models

Task Prompt

Read the fictional report briefing below. Then write a summary for local newspaper readers who have not read the report.

Requirements:

  • Length: 230 to 320 words.
  • Format: continuous prose in two to four paragraphs. Do not use bullet points or headings.
  • Tone: neutral and factual. Do not take a side on any disputed question, and do not add opinions, guesses, or facts that are not in the source.
  • The summary must cover all of the following:
    1. What happened, including when, where, how many households were affe...
Show more ▼

Read the fictional report briefing below. Then write a summary for local newspaper readers who have not read the report.

Requirements:

  • Length: 230 to 320 words.
  • Format: continuous prose in two to four paragraphs. Do not use bullet points or headings.
  • Tone: neutral and factual. Do not take a side on any disputed question, and do not add opinions, guesses, or facts that are not in the source.
  • The summary must cover all of the following:
    1. What happened, including when, where, how many households were affected, and for how long.
    2. The panel's distinction between the rupture itself and the scale of its consequences.
    3. Why the main had not been replaced, and the cost of that delay.
    4. The valve failures and the staffing and maintenance problems behind them.
    5. The main communication and support failures, plus the one area the panel praised.
    6. The panel's disagreement over funding, with both positions stated fairly.
    7. How the board responded, including which recommendation is still undecided and when it will be decided, and the compensation offered.
    8. The finding about illness and contamination.
  • Keep the key numbers accurate. You may leave out minor figures to stay within the word limit.

SOURCE PASSAGE:

Harrowgate Valley Water Authority: Independent Review of the March Distribution Failure (Executive Briefing)

At 23:40 on 14 March, a 900-millimetre trunk main ruptured at the Carrow Bend junction. Water supply was cut to about 41,000 households in the eastern districts of Linmouth and Saltreach. Depending on location, outages lasted between 19 and 63 hours. Another 12,000 households in neighbouring Ferrisford kept their supply but were placed under a precautionary boil-water notice for six days. The Authority's board commissioned an independent review panel of three people, chaired by civil engineer Dr. Imelda Okonkwo-Hart. The panel delivered its findings on 2 June. This briefing summarizes those findings and the board's initial response.

The panel found that the failed section was a cast-iron pipe laid in 1958. Over several decades it had suffered severe graphitic corrosion, which weakened the pipe wall to roughly a third of its original thickness in places. February had been the wettest month on record in the valley, with 212 millimetres of rain, almost double the long-term average. The waterlogged clay soils around Carrow Bend shifted and put extra stress on the pipe. The most likely trigger was a pressure surge caused when a pump at the Dunmere station restarted after routine maintenance, eleven minutes before the rupture. The panel concluded that the rupture itself was "not reasonably preventable in the short term" given the pipe's condition by early March. However, it judged that the scale and length of the disruption that followed were largely preventable.

On asset management, the panel noted that the Carrow Bend main was rated "high risk" in the Authority's 2017 condition survey. Its replacement was scheduled for 2020 but was postponed twice. In 2020, capital funds were reallocated during pandemic-era budget pressures. In 2022, the money was redirected to meet a new national regulatory deadline for replacing lead service pipes. The panel said each postponement was documented and defensible when viewed on its own. The problem was that the board never received a combined picture of how risk was building up across deferred projects. Replacement had been estimated at 6.8 million in 2020. The emergency replacement now under way is expected to cost 11.2 million.

The emergency response was badly hampered by valve failures. Seven isolation valves had to be closed to cut off the damaged section. Three of them would not close fully, and one had not been exercised (opened and closed for maintenance) since 2011. Engineers had expected isolation to take about two hours; it took nine. During that time the Saltreach service reservoir drained completely, which extended outages in the farthest parts of the network. The Authority's own policy is to exercise 20 percent of its valves each year, but over the past five years it averaged only 6 percent. The panel linked this shortfall directly to a 2019 decision to cut the dedicated valve maintenance crew from eight staff to five.

Communication with residents also fell short. The Authority's text-message alert system relies on customers signing up, and its register had not been properly updated. As a result, alerts reached only 38 percent of affected households, and many residents first heard about the outage through social media or neighbours. Bottled-water distribution points opened at 09:00 on 15 March, about nine hours after the rupture. The panel considered that timing acceptable, but criticized the choice of locations. Most points were in car parks of large retail sites, which suited car owners. In Saltreach, where 31 percent of households do not have a car, only one distribution point operated until the second day. Of the 1,140 customers on the Authority's register of vulnerable customers, 214 were not contacted within 24 hours. The panel did praise the customer call centre, which handled about 17,500 calls during the incident. It singled out frontline staff for their composure and for working voluntary extra shifts.

The panel did not agree on one issue: how to pay for faster replacement of ageing mains. Dr. Okonkwo-Hart and panel member Rafael Dunstan, a utility finance specialist, support a ring-fenced surcharge of about 4 percent on customer bills for ten years, to be used only for main replacement. The third member, consumer advocate Petra Lindqvist, dissented. She argued that customers should not pay more until the Authority has redirected money already in its own budget, especially the 9.5 million set aside for refurbishing its headquarters, and has applied for national infrastructure grants. The report records both positions in full and does not resolve the disagreement.

The panel made five recommendations. First, a consolidated risk register covering all deferred capital projects should be reported to the board every quarter. Second, valve exercising should return to the 20 percent annual target, with the valve crew restored to eight staff within twelve months. Third, the Carrow Bend corridor and two other high-risk mains, at Pellow Road and Ashby Cross, should be replaced within four years. Fourth, text alerts should switch from an opt-in system to an opt-out system based on billing records. Fifth, during future outages, every affected household should be within one kilometre walking distance of a water distribution point, and vulnerable customers should be contacted within six hours.

The board accepted recommendations one, two, four and five in full. It accepted recommendation three only "in principle", because how to fund it is still undecided. That decision is due in September, after a public consultation. Chief executive Owen Raddle publicly apologized to affected residents. He confirmed compensation of 45 per household for any outage longer than 24 hours, which is above the regulatory minimum of 30. Finally, the panel found no evidence that the incident contaminated drinking water or caused any illness. It described the Ferrisford boil-water notice as a justified precaution, not a response to confirmed contamination.

Task Context

The source passage is entirely fictional and was written specifically for this task. Any resemblance to real organizations, places, or people is coincidental.

Judging Policy

A strong answer is 230 to 320 words of neutral prose in two to four paragraphs, with no bullets or headings. It covers all eight required elements accurately: the incident basics (14 March, Carrow Bend, about 41,000 households in Linmouth and Saltreach, outages of 19 to 63 hours, and ideally the Ferrisford boil-water notice); the panel's view that the rupture was not reasonably preventable in the short term but the scale of disruption largely was; the two deferrals and their reasons (pandemic-era budget reallocatio...

Show more ▼

A strong answer is 230 to 320 words of neutral prose in two to four paragraphs, with no bullets or headings. It covers all eight required elements accurately: the incident basics (14 March, Carrow Bend, about 41,000 households in Linmouth and Saltreach, outages of 19 to 63 hours, and ideally the Ferrisford boil-water notice); the panel's view that the rupture was not reasonably preventable in the short term but the scale of disruption largely was; the two deferrals and their reasons (pandemic-era budget reallocation and the lead-pipe deadline), the missing combined risk picture, and the cost rising from 6.8 million to 11.2 million; three of seven valves failing, isolation taking nine hours instead of two, valve exercising averaging 6 percent against a 20 percent target, and the crew cut from eight to five; the alert reach of 38 percent, poorly located distribution points disadvantaging people without cars, 214 vulnerable customers not contacted within 24 hours, and praise for the call centre; the funding split, with the 4 percent ten-year surcharge on one side and Lindqvist's dissent (redirect the 9.5 million headquarters budget and seek grants first) on the other, presented even-handedly; the board accepting four recommendations in full, recommendation three only in principle with a funding decision in September after consultation, and 45 per household compensation versus the 30 minimum; and no evidence of contamination or illness, with the boil notice described as precautionary. Penalize factual errors, wrong numbers, invented details, taking sides on the funding dispute, confusing the rupture's preventability with that of its consequences, leaving out required elements, ignoring the word limit, or using bullet format. Prefer answers that are clear, well organized, and readable for a general audience over ones that simply compress sentences from the source.

Answers

On 14 March, a trunk main ruptured at Carrow Bend, cutting water to about 41,000 households in eastern Linmouth and Saltreach for 19 to 63 hours. Another 12,000 households in Ferrisford faced a precautionary six-day boil-water notice. An independent panel found that corrosion, waterlogged ground and a likely pressure surge contributed to the break. It judged the rupture not reasonably preventable in the short term, but said the scale and duration of the disruption were largely preventable. The high-risk main was due for replacement in 2020, but funds were diverted amid pandemic-era pressures and later to meet a l...

Show Full Answer ▼

On 14 March, a trunk main ruptured at Carrow Bend, cutting water to about 41,000 households in eastern Linmouth and Saltreach for 19 to 63 hours. Another 12,000 households in Ferrisford faced a precautionary six-day boil-water notice. An independent panel found that corrosion, waterlogged ground and a likely pressure surge contributed to the break. It judged the rupture not reasonably preventable in the short term, but said the scale and duration of the disruption were largely preventable. The high-risk main was due for replacement in 2020, but funds were diverted amid pandemic-era pressures and later to meet a lead-pipe deadline. Each delay was defensible individually, the panel said, but the board lacked a combined view of accumulating risk.

Isolation took nine hours rather than the expected two after three of seven valves failed to close fully. One had not been exercised since 2011. Annual valve maintenance averaged 6 percent against a 20 percent target, a shortfall the panel linked to cutting the dedicated crew from eight staff to five. The delay drained Saltreach’s reservoir, extending outages. Text alerts reached only 38 percent of affected households; water distribution sites favoured car owners, and 214 of 1,140 registered vulnerable customers were not contacted within 24 hours. The panel praised call-centre staff, who handled about 17,500 calls.

Two panel members proposed a ring-fenced bill surcharge of about 4 percent for ten years to accelerate main replacement. The dissenting member argued that existing funds, including 9.5 million earmarked for headquarters refurbishment, and national grants should be pursued before customers pay more. Replacement, estimated at 6.8 million in 2020, is now expected to cost 11.2 million. The board accepted four recommendations fully, but accepted replacement of three high-risk mains only in principle; funding will be decided in September after public consultation. It offered 45 per household for outages exceeding 24 hours. The panel found no evidence of drinking-water contamination or illness.

Result

#1 | Winner

Winning Votes

3 / 3

Average Score

84

Total Score

83

Overall Comments

Answer A is a remarkably comprehensive summary that successfully incorporates almost every specific metric and requirement from the prompt while adhering to all format constraints (310 words, three paragraphs, neutral prose). Its only slight weakness is structural: it places the replacement cost figures (6.8 million rising to 11.2 million) in the third paragraph alongside the funding dispute, rather than in the first paragraph with the discussion of the deferred replacement.

View Score Details ▼

Faithfulness

Weight 40%
85

Faithful and accurate to the source. It captures the panel's critical distinction accurately ('not reasonably preventable in the short term').

Coverage

Weight 20%
88

Covers all eight required points with impressive precision, preserving almost all key metrics specified in the prompt.

Compression

Weight 15%
85

Demonstrates exceptional density and concision, packing extensive factual detail into 310 words without losing readability.

Clarity

Weight 15%
75

Clear, objective, and accessible for local newspaper readers, though the abrupt insertion of replacement costs in paragraph three slightly interrupts the flow.

Structure

Weight 10%
70

Well-formatted into three continuous prose paragraphs without headings or bullets. The placement of the 2020/current cost figures in paragraph three feels slightly disconnected from the delay explanation in paragraph one.

Total Score

88

Overall Comments

Answer A is an accurate, neutral and comprehensive summary that meets the length and paragraph requirements. It preserves important qualifications, explains the deferred replacement and maintenance failures, and presents the funding disagreement fairly. Minor omissions include the alert system's shortcomings, the four-year replacement timetable and the regulatory compensation minimum. The replacement cost comparison would fit more naturally alongside the explanation of the deferrals.

View Score Details ▼

Faithfulness

Weight 40%
92

Preserves the distinction between a rupture not reasonably preventable in the short term and consequences that were largely preventable. It correctly treats the pressure surge as likely, retains the panel's finding that individual deferrals were defensible, and accurately reports the key numbers and competing funding positions.

Coverage

Weight 20%
87

Covers all eight required areas with substantial supporting detail, including cumulative risk, replacement costs, valve maintenance rates, crew sizes, vulnerable-customer figures, the ten-year surcharge and consultation before September's decision. It omits the alert system's underlying problems, the four-year replacement deadline and the compensation minimum.

Compression

Weight 15%
86

Fits a notably broad selection of relevant facts into the 230–320-word limit without bullets or excessive shorthand. Almost every sentence contributes directly to a required topic, although some lower-priority figures could have been traded for the alert-system explanation.

Clarity

Weight 15%
82

Uses accessible language and clearly connects staffing cuts, valve failures, reservoir drainage and longer outages. The final paragraph is dense, and placing the replacement cost comparison there separates it from the deferral explanation.

Structure

Weight 10%
83

Uses three continuous-prose paragraphs with no headings or bullets, moving logically from the incident and asset management to emergency response and then funding and board action. The final paragraph combines several distinct topics but remains coherent.

Total Score

81

Overall Comments

Answer A is a tight, accurate summary that preserves nearly every key figure from the source (41,000 households, 19 to 63 hours, 12,000 under a six-day notice, three of seven valves, nine hours versus two, 6 percent versus 20 percent, crew cut from eight to five, 38 percent alert reach, 214 of 1,140 vulnerable customers, 17,500 calls, 4 percent for ten years, 9.5 million headquarters budget, 6.8 to 11.2 million, September decision after consultation, 45 compensation). It correctly keeps the nuance that the rupture was not reasonably preventable in the short term while the disruption was largely preventable, and it captures the panel's systemic finding that the board lacked a combined picture of deferred risk. The funding dispute is stated even-handedly. Minor gaps: it omits the 30 regulatory minimum, the chief executive's apology, and does not explicitly restate that the boil notice was judged a justified precaution in the illness finding (though it calls the notice precautionary earlier). The cost-rise sentence sits a little awkwardly in the funding paragraph, and at roughly 316 words it is close to the ceiling.

View Score Details ▼

Faithfulness

Weight 40%
85

All figures and attributions check out against the source; nuances such as 'not reasonably preventable in the short term' and 'largely preventable' are preserved, the two deferral reasons are correct, and the funding dispute is presented neutrally with both positions accurately paraphrased. No invented details. Only small omissions (30 minimum, apology) rather than errors.

Coverage

Weight 20%
85

Covers all eight required elements with their key numbers, including the combined-risk-picture finding, 6 vs 20 percent valve exercising, crew cut from eight to five, 214 of 1,140 vulnerable customers, call centre praise, 9.5 million headquarters budget, September decision after consultation, 45 compensation, and the no-contamination finding. Omits the 30 regulatory minimum and the apology.

Compression

Weight 15%
75

About 316 words, inside the limit though near the ceiling. Word budget is spent on the most decision-relevant figures and findings, with minor items (exact time, pipe diameter, rainfall) sensibly dropped.

Clarity

Weight 15%
75

Sentences are direct and understandable for general readers; cause-and-effect links (valve failures leading to reservoir draining and longer outages; crew cut leading to maintenance shortfall) are explicit. Dense in places, and the cost-rise sentence appears somewhat out of context in the funding paragraph.

Structure

Weight 10%
75

Three balanced paragraphs with a logical progression: incident and causes, response failures, funding dispute and board response. Only flaw is the cost figure placed in the funding paragraph rather than with the deferral history.

An independent review panel examined the March water distribution failure that occurred when a 900-millimetre trunk main ruptured at the Carrow Bend junction at 23:40 on 14 March. The incident cut water supply to about 41,000 households in Linmouth and Saltreach for between 19 and 63 hours, while another 12,000 households in Ferrisford faced a six-day boil-water notice. The panel concluded the rupture itself was not reasonably preventable due to severe graphitic corrosion, weather stress, and a pressure surge from a restarted pump, but the scale and length of the disruption were preventable. Replacement of the pi...

Show Full Answer ▼

An independent review panel examined the March water distribution failure that occurred when a 900-millimetre trunk main ruptured at the Carrow Bend junction at 23:40 on 14 March. The incident cut water supply to about 41,000 households in Linmouth and Saltreach for between 19 and 63 hours, while another 12,000 households in Ferrisford faced a six-day boil-water notice. The panel concluded the rupture itself was not reasonably preventable due to severe graphitic corrosion, weather stress, and a pressure surge from a restarted pump, but the scale and length of the disruption were preventable. Replacement of the pipe had been postponed twice from its original 2020 schedule due to pandemic budget pressures and lead service pipe regulations, causing estimated replacement costs to rise from 6.8 million to 11.2 million.

Emergency response was delayed by valve failures because three of seven isolation valves failed to close fully and one had not been exercised since 2011, stretching isolation time to nine hours. The shortfall was linked to a 2019 staff cut in the maintenance crew. Communication also fell short as text alerts reached only 38 percent of homes, bottled-water sites were poorly located for non-car owners, and vulnerable customers missed timely contact. However, the panel praised the call centre and frontline staff. The panel disagreed on funding faster replacements: two members favoured a ring-fenced 4 percent bill surcharge, while consumer advocate Petra Lindqvist dissented, urging budget reallocation from headquarters refurbishment and national grants instead.

The board accepted four recommendations fully, but accepted the third recommendation regarding main replacements only in principle pending a September funding decision. Chief executive Owen Raddle apologized and offered 45 compensation to households facing outages over 24 hours. Finally, the panel found no evidence of contamination or illness, confirming the Ferrisford notice was purely a precaution.

Result

#2

Winning Votes

0 / 3

Average Score

70

Total Score

76

Overall Comments

Answer B is written in smooth, well-organized prose and fits the word limit cleanly (310 words). However, it omits numerous key specific figures required by the prompt, such as the target versus actual valve maintenance rates (20 percent vs. 6 percent), the staffing reduction numbers (cut from eight to five), the vulnerable customer numbers (214 not contacted within 24 hours), the duration of the proposed surcharge (ten years), and the specific headquarters budget amount (9.5 million).

View Score Details ▼

Faithfulness

Weight 40%
80

Accurate overall, though it drops the key qualifying phrase 'in the short term' when describing whether the rupture was preventable.

Coverage

Weight 20%
65

Covers the general topics but omits several essential figures: the valve exercising rates (6% vs 20%), the staff cut size (8 to 5), the vulnerable customer figures (214 of 1,140 within 24 hours), the surcharge timeframe (10 years), and the 9.5 million headquarters budget.

Compression

Weight 15%
75

Good compression at 310 words, but achieves its brevity by omitting required factual metrics rather than tighter sentence formulation.

Clarity

Weight 15%
80

Very clear, lucid, and easy to read, with natural sentence flow throughout.

Structure

Weight 10%
80

Exemplary three-paragraph structure with logical thematic grouping, keeping the cost of delay immediately next to the explanation of the delay.

Total Score

71

Overall Comments

Answer B meets the format and length requirements and covers all eight broad topics in readable prose. However, it removes important qualifications from the panel's findings about preventability and the suspected pressure surge. It also omits several substantive details, particularly the missing cumulative risk assessment, maintenance targets and staffing numbers, the surcharge duration and public consultation.

View Score Details ▼

Faithfulness

Weight 40%
70

Most facts and numbers are accurate, but the rupture finding loses its short-term qualification, the consequences become unqualifiedly preventable, and the pressure surge is presented as established rather than the most likely trigger. These changes overstate the panel's conclusions.

Coverage

Weight 20%
65

Covers the incident, deferrals, valves, communications, funding split, board response and health finding. However, it omits the missing combined risk picture, the two-hour isolation expectation, maintenance rates and crew sizes, vulnerable-customer figures, the surcharge's ten-year duration, the headquarters budget amount and public consultation.

Compression

Weight 15%
71

Stays within the word limit and avoids repetition, but its selection is less efficient: pipe diameter, exact rupture time and technical corrosion terminology receive space while more important explanatory details and funding terms are omitted.

Clarity

Weight 15%
76

Generally readable and easy to follow, with clear transitions between major topics. Unexplained graphitic corrosion terminology is less suitable for general readers, and references to the third recommendation and untimely vulnerable-customer contact are somewhat vague.

Structure

Weight 10%
80

Uses three compliant prose paragraphs and broadly follows the report's progression. The funding dispute is appended to the emergency-response paragraph, making the thematic grouping slightly less cohesive.

Total Score

64

Overall Comments

Answer B is readable and within the word limit, and it includes some details A lacks (the 23:40 time, 900-millimetre pipe, the chief executive's apology, panel members' names). However, it spends words on these lower-value details while dropping more important ones: the 20 percent target versus 6 percent actual, the crew cut from eight to five, the 214 uncontacted vulnerable customers, the 9.5 million headquarters figure, the ten-year duration of the surcharge, the public consultation, and above all the panel's central systemic finding that the board never saw a combined picture of accumulating risk. It also softens nuance: it drops 'in the short term' and 'largely' from the preventability finding, states that postponements 'caused' the cost rise (an inference), frames Lindqvist's position as 'instead' rather than 'until', and refers to 'the shortfall' without having described any shortfall. Structurally, the funding dispute is tacked onto the end of the response-failures paragraph and the final paragraph is thin.

View Score Details ▼

Faithfulness

Weight 40%
65

No outright wrong numbers, but several nuance losses and mild inferences: 'in the short term' and 'largely' are dropped from the preventability finding, postponements are said to have 'caused' the cost rise, Lindqvist's conditional 'until' position is rendered as 'instead', 'the shortfall' is referenced without being described, and 'confirming... purely a precaution' slightly overstates the panel's wording.

Coverage

Weight 20%
60

Touches all eight elements but thinly: omits the board's missing combined risk picture, the 6 vs 20 percent maintenance statistics, the crew size change, the 214 vulnerable customers figure, the 9.5 million figure, the ten-year surcharge duration, the public consultation, and the 30 minimum. Includes the apology and names but these are less central than what was dropped.

Compression

Weight 15%
65

About 290 words, comfortably within limit, but allocates words to low-value specifics (23:40, 900-millimetre, graphitic corrosion, personal names) while cutting the maintenance statistics, the systemic risk finding, and key monetary figures that readers would need.

Clarity

Weight 15%
65

Generally readable, but 'The shortfall was linked to a 2019 staff cut' refers to a shortfall never introduced, several sentences are long and stacked with clauses, and 'vulnerable customers missed timely contact' is vague compared with the source.

Structure

Weight 10%
65

Three paragraphs in prose, but the funding dispute is appended to the end of the response-failures paragraph rather than grouped with the board's funding decision, and the final paragraph is noticeably short, giving an unbalanced feel.

Comparison Summary

Final rank order is determined by judge-wise rank aggregation (average rank + Borda tie-break). Average score is shown for reference.

Judges: 3

Winning Votes

3 / 3

Average Score

84
View this answer

Winning Votes

0 / 3

Average Score

70
View this answer

Judging Results

Why This Side Won

Answer A wins on the heavily weighted faithfulness criterion by preserving the source's nuances (short-term non-preventability, 'largely' preventable consequences, the 'until' framing of the dissent) and by introducing no inferences, whereas B drops qualifiers, adds a causal claim about costs, and uses an unanchored 'the shortfall' reference. A also clearly leads on coverage, including the missing combined risk picture, the maintenance statistics, the 214 vulnerable customers, the 9.5 million figure, and the public consultation that B omits, and it uses its word budget more effectively. The weighted result favors A decisively.

Why This Side Won

Answer A wins on the weighted criteria because it is more faithful to the panel's qualified conclusions and retains substantially more of the information needed to explain the outage's consequences and the funding dispute. It achieves this within the required length and format, without sacrificing neutrality or readability.

Why This Side Won

Answer A wins primarily due to its superior coverage and compression. While both answers hit exactly 310 words and adhere to the formatting and neutral tone guidelines, Answer A manages to include virtually all specific numbers and factual nuances specified in the task prompt (such as maintenance targets, staffing reductions, vulnerable customer figures, and surcharge terms). Answer B glosses over several of these required details with vague phrasing.

X f L