01Overview
This project compares presidential results across rounds and elections to identify deviations from historical and contextual models, quantify uncertainty and investigate whether geographical patterns recur. Version 0.3 retains the assessment of the two reviews and incorporates the subsequent counter-analysis, strengthening four operating rules.
Both rounds of the 2010, 2014, 2018, 2022 and 2026 elections.
The municipal pilot covers Rio de Janeiro municipalities where data coverage passes validation.
The first phase covers municipalities. The second expands to polling places and comparisons across states.
Identify results that are statistically atypical under explicit, validated models, measure their magnitude and uncertainty, check for concentration within municipalities and test whether the results recur in comparable municipalities within and across states.
Araruama, Resende and Volta Redonda motivated the initial exploration. Their results have already been examined, so these municipalities cannot serve as blind cases or independent confirmatory evidence. Any classification as atypical still requires completed and validated analysis.
The first empirical run reconciled official data from all five elections, checked the three 2026 examples and evaluated a historical baseline on 2022 data. Confirmatory analysis of 2026 requires further calibration.
The matrix of 21 decisions records how the project assessed the reviews. The counter-analysis section documents the four points adopted and the changes made in v0.3. The sidebar contents and earlier numerical examples remain available.
02Research questions
- How did the vote count for each finalist change between the first and second rounds of each election?
- How do these gains or losses compare with the votes cast for candidates eliminated in the first round?
- How much of the change in valid votes can be accounted for by changes in the electorate, abstention, blank votes and invalid votes?
- Are these changes consistent with the unit's history and with patterns in municipalities with similar characteristics?
- Are the 2026 deviations widespread or concentrated in particular polling places?
- Do municipalities with shared characteristics show similar patterns, first within the same state and then across states?
The analysis will cover both finalists in each election. It may examine deviations in different directions, including gains, losses, changes in participation and combinations of indicators.
03Scope and units of analysis
Office · President of the Republic. Elections · 2010, 2014, 2018, 2022 and 2026. The first phase covers all 92 municipalities in Rio de Janeiro state, extending the study beyond the three cases that prompted the question. The municipality count comes from the Regional Electoral Court of Rio de Janeiro (TRE-RJ). [D7]
Municipalities as the main unit and polling places for additional analysis
| Criterion | Municipality | Polling place |
|---|---|---|
| Historical continuity | Generally easier to establish. Check boundaries and electoral records. | Check for changes in address, code and the set of polling sections. |
| Changes within the unit | Reassignments within the municipality preserve its total. | Reassignments within the polling place preserve its total. |
| Sensitivity to concentrated deviations | May dilute deviations at a small number of polling places. | Can reveal concentrations within a municipality. |
| Variability and preparation | Small counts have less influence, and the data are easier to integrate. | Small polling places show greater fluctuations, and historical integration requires matching records across elections. |
| Role in the project | Main unit in the historical panel. | Additional unit for analysing patterns within municipalities. |
Phase I · Municipal analysis. Ingest and reconcile official data, fit a parsimonious model, validate it on historical data and assess all eligible municipalities in Rio de Janeiro. The spatial structure will use municipalities. Nationwide matching of polling places is no longer a prerequisite for the pilot.
Phase II · Polling places and comparisons across states. Run descriptive screening and, where comparable historical data exist, assess polling places in every eligible pilot municipality, regardless of whether the municipality was flagged. Reserve in-depth documentary work and additional modelling for selected priorities and controls. Use other states for external validation, keeping exploratory adaptations separate from genuinely held-out tests.
Municipal flags will inform priorities for in-depth investigation. All eligible polling places will still enter screening. Deviations within municipalities can offset one another or be diluted in the municipal total. Fix the number and selection of controls for in-depth investigation before examining their polling-place results.
Retain polling sections as the source unit. Changes within a polling place or municipality can preserve its aggregate total. Transfers across units require reconciliation. Nearby buildings may serve different voters, and polling-place boundaries need not align with neighbourhoods.
Changes in scale affect results and correlations. Compare municipal and polling-place findings, treating records derived from the same voting machines as dependent. [R2]
Polling-place coverage independent of municipal results
- Cross-sectional screening. Include all pilot polling places with reconciled data for the period under analysis, even where the historical match is insufficient. Describe rates and changes between comparable rounds. Record gaps in historical data separately from results within expected ranges.
- Historical comparison. Include all polling places whose matches and data coverage meet predefined quality criteria. Eligibility is based on data quality and historical matching. It is independent of which candidate gained, the size of the gain ratio and municipal statistical significance.
- In-depth investigation. Prioritise flagged patterns and controls for technical and documentary analysis. Record the basis for targeted selection.
- Resource constraints. If full polling-place coverage is infeasible, report partial coverage. A sampling alternative requires a probability design, potentially stratified, with known inclusion probabilities and stated limits on generalisation. Disclose any departure from planned coverage, and distinguish screening from a list of extreme cases.
This work may follow the municipal pilot to fit operational capacity. The requirement governs how polling places are selected. The municipal panel can begin before full harmonisation of polling-place records.
04Data, governance and electoral register reconciliation
Required empirical source. Use primary files from the TSE Open Data Portal for results, vote count details, electorate, polling places, aggregates and transfers. Verify package and field names against each year's data dictionary, allowing for schema changes. Supplement municipal profiles with IBGE data.
Retain the secondary sources from v0.1 as a record of exploratory work and a point of comparison. The models will use validated primary data. Protocol preparation can begin before ingestion is complete. Freezing the confirmatory protocol requires the data quality report and an inventory of results already examined.
Traceability and reproducible execution
- Preserve original files, URLs, filenames, licences, retrieval and generation dates, versions and SHA-256 hashes.
- Keep original files, processed data, indicators, model fitting, validation and reporting in separate layers.
- Record data dictionaries, reconciliation checks, exclusions, missing data and classification changes.
- Version scripts, the environment, dependencies and random seeds. Provide a reproducible container configuration and instructions for running it.
- Make materials available as permitted by their licences and provide a manifest sufficient for reproduction. Exclude unnecessary personal data from publication.
The hash verifies the identity of the preserved bytes, and the container supports a reproducible environment. Both support auditing the study. Establishing whether electronic records match each voter's intent requires further evidence.
Electoral populations and special categories
For each round, document the reference electorate, the people eligible to vote in the unit, actual turnout and the vote categories. Distinguish voters' original registration, temporary eligibility and polling destination as applicable to the office.
- Voting away from the registered location and temporary transfers. Check origins and destinations to avoid double counting. The data must reflect the population eligible for the particular office and round. [D9]
- Special polling sections. Identify and retain each category and its population. Justify and test any aggregation.
- Notifications of absence. Treat these as records of absence. Exclude voters who notified an absence from turnout and vote counts. [D10]
- Difference E₂ − E₁. Investigate differences using official data versions and definitions. Voting away from the registered location, transfers and adjustments are testable hypotheses. The causes of the differences in the three examples remain unverified.
Historical matches and confidence levels
Combine codes, names, addresses, coordinates, section composition and change records. Treat coordinates and IBGE boundary data as supporting evidence because a nearby new building may serve a different group of voters. Allow a match to remain unresolved if proximity is the only evidence.
Classify matches as confirmed, probable or unresolved using documented criteria. Review ambiguous cases manually, repeat the analyses using the most reliable subset and describe the excluded units. Distinguish polling-place addresses from voters' home addresses.
Criteria for using historical matches
| Class | Evidence and use |
|---|---|
| A · Documented | Administrative records and polling-section composition support the match, with changes documented and populations reconciled. Include the match in the main historical comparison if it meets the other requirements. |
| B · Probable | Partial evidence includes names, addresses and coordinates. Documentation is insufficient to establish continuity. Include these matches separately in sensitivity analysis. |
| C · Unresolved | Composition conflicts, splits, mergers or insufficient evidence prevent a match. Keep the match unresolved and include the unit in cross-sectional screening if its current data are usable. |
These classes describe administrative continuity at unit level. Individual voter identity is outside their scope. Changes in the electorate over the 16 years will be interpreted alongside changes in the registration profile.
Proximity alone cannot promote a match from B or C to A. Retain the evidence, the rule applied, any necessary manual decision and the date of each match. For a documented split or merger, consider a stable aggregation across the historical series and specify how the unit of analysis changes.
Report the number of polling places and electorate covered in each class by municipality and election, together with reasons for exclusion. Compare the profiles of matched and unmatched units. Restrict conclusions about the absence of anomalies to the units assessed.
05Panel indicators
Calculate indicators for each unit, year and round. Analyse transitions for the two finalists in each election, grouping all other first-round candidates under ‘other candidates’.
| Group | Indicators and denominators |
|---|---|
| Participation | Electorate (E), turnout (C) and abstention (A), with rates C/E and A/E. |
| Vote validity | Valid votes (V), blank votes (B), invalid votes (N) and any other required categories, with rates calculated using C. |
| Votes for finalists | Vote count and share of valid votes for each candidate in each round. |
| Other first-round candidates | Vote count and share of valid votes for candidates who did not reach the second round. |
| Changes between rounds | Absolute differences, relative changes and changes in rates in percentage points. |
| Gains relative to other candidates' votes | Each finalist's ΔF / O₁, as a percentage, and the sum of the gains / O₁. |
| Common denominator | Δ of each count / E₁, with additive decomposition of valid votes and combined gains. |
| Model | Predictions, prediction intervals, residuals and joint measures of deviation, with uncertainty. |
Finalists' gains relative to other candidates' votes
This measure is called gain relative to the first-round total for other candidates. It can be negative or exceed 100%. It relates aggregate changes. The data do not identify individual transfers. If O1 is zero, the ratio is undefined. Flag instability when the denominator is very small and display the counts.
Use the same definitions in every election. Comparisons by party or political bloc require a separate rule defined in advance, since candidates, alliances and opponents change across elections.
06Accounting reconciliation
The accounting identities help check the consistency of the data and decompose changes before statistical modelling.
Electorate, participation and valid votes
If a separate, mutually exclusive category X is needed to reconcile turnout, the identity is V = E − A − B − N − X. Check the fields against the dictionary for each file and subtract each category only once.
Combined gains of the finalists
A fall in abstention, blank votes or invalid votes adds to the net change in valid votes. A rise reduces it. This decomposition describes aggregate counts. The source of each candidate's gains remains unknown at voter level.
The identity also implies that R1 + R2 = 100 × (1 + ΔV/O1). The sum of the ratios can therefore exceed 100% when valid votes increase.
Reconciliation checks internal consistency. An independent test of electoral integrity requires separate evidence. Several indicators are mathematically related, so account for their dependence and avoid combinations that produce exact collinearity.
07Percentage comparisons
The project uses three complementary measures. State each column's denominator so that municipalities of different sizes can be compared.
Change relative to the indicator itself
Contribution on a common denominator
This normalisation allows contributions to be compared across municipalities. Express them as a percentage of the first-round electorate or as votes per thousand electors.
Change in rates in percentage points
The analysis also compares rates between rounds. The change in the abstention rate in percentage points is 100 × (A2/E2 − A1/E1). Candidate shares use V as their denominator. Rates for valid, blank and invalid votes use C. Abstention and turnout rates use E.
Relative change in a count, a change in rates in percentage points and a contribution relative to E1 answer different questions. A large percentage reduction in blank votes, for example, may represent only a small share of the electorate.
08Municipal examples
All 54 counts in the three tables matched the TSE's official files. The municipalities remain exploratory cases.
In all three tables, candidate shares, including the share for ‘other candidates’, use valid votes. Rates for valid, blank and invalid votes use turnout. Abstention and turnout rates use the electorate. The change column uses the first-round count as its denominator. The electorate is shown as 100% in each round.
Araruama (RJ)
| Indicator | First round | Second round | Absolute and relative change |
|---|---|---|---|
| Flávio Bolsonaro (PL) | 53,415 (64.98%) | 53,426 (64.79%) | +11 (+0.02%) |
| Lula (PT) | 23,451 (28.53%) | 29,032 (35.21%) | +5,581 (+23.80%) |
| Other candidates | 5,331 (6.49%) | — | — |
| Valid votes | 82,197 (95.54%) | 82,458 (96.77%) | +261 (+0.32%) |
| Blank votes | 1,655 (1.92%) | 981 (1.15%) | −674 (−40.73%) |
| Invalid votes | 2,179 (2.53%) | 1,768 (2.07%) | −411 (−18.86%) |
| Abstention | 25,339 (22.75%) | 26,161 (23.49%) | +822 (+3.24%) |
| Turnout | 86,031 (77.25%) | 85,207 (76.51%) | −824 (−0.96%) |
| Electorate | 111,370 (100.00%) | 111,368 (100.00%) | −2 (−0.00%) |
Resende (RJ)
| Indicator | First round | Second round | Absolute and relative change |
|---|---|---|---|
| Flávio Bolsonaro (PL) | 43,226 (59.81%) | 45,289 (61.08%) | +2,063 (+4.77%) |
| Lula (PT) | 22,536 (31.18%) | 28,858 (38.92%) | +6,322 (+28.05%) |
| Other candidates | 6,515 (9.01%) | — | — |
| Valid votes | 72,277 (95.69%) | 74,147 (96.56%) | +1,870 (+2.59%) |
| Blank votes | 1,507 (2.00%) | 994 (1.29%) | −513 (−34.04%) |
| Invalid votes | 1,752 (2.32%) | 1,644 (2.14%) | −108 (−6.16%) |
| Abstention | 22,484 (22.94%) | 21,238 (21.67%) | −1,246 (−5.54%) |
| Turnout | 75,536 (77.06%) | 76,785 (78.33%) | +1,249 (+1.65%) |
| Electorate | 98,020 (100.00%) | 98,023 (100.00%) | +3 (+0.00%) |
Volta Redonda (RJ)
| Indicator | First round | Second round | Absolute and relative change |
|---|---|---|---|
| Flávio Bolsonaro (PL) | 82,586 (50.06%) | 85,512 (50.42%) | +2,926 (+3.54%) |
| Lula (PT) | 69,383 (42.06%) | 84,089 (49.58%) | +14,706 (+21.20%) |
| Other candidates | 12,989 (7.87%) | — | — |
| Valid votes | 164,958 (95.68%) | 169,601 (96.83%) | +4,643 (+2.81%) |
| Blank votes | 3,284 (1.90%) | 1,947 (1.11%) | −1,337 (−40.71%) |
| Invalid votes | 4,167 (2.42%) | 3,610 (2.06%) | −557 (−13.37%) |
| Abstention | 49,548 (22.32%) | 46,820 (21.09%) | −2,728 (−5.51%) |
| Turnout | 172,409 (77.68%) | 175,158 (78.91%) | +2,749 (+1.59%) |
| Electorate | 221,957 (100.00%) | 221,978 (100.00%) | +21 (+0.01%) |
Finalists' gains relative to other candidates' votes
| Municipality | Others · first round | Δ Flávio | Δ Flávio / others | Δ Lula | Δ Lula / others |
|---|---|---|---|---|---|
| Araruama | 5,331 | +11 | 0.21% | +5,581 | 104.69% |
| Resende | 6,515 | +2,063 | 31.67% | +6,322 | 97.04% |
| Volta Redonda | 12,989 | +2,926 | 22.53% | +14,706 | 113.22% |
Breakdown of the change in valid vote counts
| Municipality | Δ electorate | −Δ abstention | −Δ blank votes | −Δ invalid votes | Δ valid votes |
|---|---|---|---|---|---|
| Araruama | −2 | −822 | +674 | +411 | +261 |
| Resende | +3 | +1,246 | +513 | +108 | +1,870 |
| Volta Redonda | +21 | +2,728 | +1,337 | +557 | +4,643 |
The abstention, blank vote and invalid vote columns represent −ΔA, −ΔB and −ΔN. Each sign shows the direction of the contribution to ΔV.
Waterfall charts for the examples
The bars start with the first-round votes for the other candidates and add the contributions from the electorate, abstention, blank votes and invalid votes to reach the finalists' combined gain. A common denominator and a vertical scale of 0 to 100 votes per thousand electors allow comparisons between municipalities. The charts use the same checked counts as the tables.
These charts show an identity between net changes. Individual voters who switched candidates or began voting cannot be identified from the bars. A Sankey diagram of individual transfers would require observed or estimated flows.
Percentage change in each count
| Municipality | Electorate | Abstention | Blank votes | Invalid votes | Valid votes |
|---|---|---|---|---|---|
| Araruama | −0.002% | +3.24% | −40.73% | −18.86% | +0.32% |
| Resende | +0.003% | −5.54% | −34.04% | −6.16% | +2.59% |
| Volta Redonda | +0.009% | −5.51% | −40.71% | −13.37% | +2.81% |
Breakdown as a percentage of the initial electorate
| Municipality | ΔE / E₁ | −ΔA / E₁ | −ΔB / E₁ | −ΔN / E₁ | ΔV / E₁ |
|---|---|---|---|---|---|
| Araruama | −0.002% | −0.738% | +0.605% | +0.369% | +0.234% |
| Resende | +0.003% | +1.271% | +0.523% | +0.110% | +1.908% |
| Volta Redonda | +0.009% | +1.229% | +0.602% | +0.251% | +2.092% |
The same denominator E1 applies to every column. Three decimal places make small changes in the electorate visible. Rounding may affect the sum of the displayed values.
In Volta Redonda, for example, the finalists gained a combined 17,632 votes. The total consists of 12,989 votes for other candidates in the first round, a net increase of 21 in the electorate, 2,728 fewer abstentions, 1,337 fewer blank votes and 557 fewer invalid votes. These aggregate figures do not establish how individuals voted.
Secondary sources for the examples include Araruama, Resende and Volta Redonda. The data will be validated against primary sources before they are used for inference.
09Core, spatial and transition models
The analysis starts with a parsimonious model and simple benchmarks. Additional complexity is warranted if it improves calibration, out-of-sample prediction and the ability to detect specified simulated perturbations. A percentage for discriminatory power has yet to be established.
Core compositional model for counts
The mutually exclusive categories in one round form a composition when divided by the same E. In the first round, the categories are the two finalists, other candidates, blank votes, invalid votes, abstention and any additional categories required. In the second round, the other-candidates category is structurally absent and is excluded from the active categories. Rates with different denominators and signed gains between rounds are analysed separately from the components of a single composition.
The models will condition second-round outcomes on first-round indicators, historical data and a small set of contextual covariates. Regularising priors, comparison with simple benchmarks and external validation will limit overfitting. The election counts cover the full population. The relevant uncertainty concerns the behavioural model and its assumptions.
ALR and ILR may be used in sensitivity analyses for positive proportions. CLR retains a sum constraint and leaves some singularity. Log-ratios cannot be applied directly to negative gains or zeros. Any pseudocounts must be disclosed. Count models must distinguish observed zeros from structural zeros. [R4]
Stabilisation while retaining the counts
Adopt hierarchical effects with partial pooling for rate and probability parameters, so estimates for small units can draw on comparable groups. Always report counts, raw rates, regularised estimates and intervals. The ratio ΔF/O₁ remains descriptive. Its uncertainty will come from the joint model, with the ratio itself left unsmoothed.
Shrinkage can reduce extremes caused by variability and can also attenuate genuine departures. Compare results with and without shrinkage and assess sensitivity to priors. The ratio remains undefined when O₁ is zero.
Explicit spatial dependence
Construct W from the IBGE municipal boundaries, defining neighbours as municipalities that share a boundary. Document the version, symmetry, connections and units with no neighbours. Prespecify k-nearest neighbours as an alternative and justify the treatment of disconnected units.
Compare the hierarchical model without a spatial structure with a BYM2 extension combining structured and unstructured spatial effects. Document the scale and priors. This structure models dependence. Distinguishing legitimate political change from manipulation requires additional evidence. Assess how much smoothing absorbs collective patterns and whether prediction improves in held-out spatial blocks. [R5]
Operational controls and turnout shocks
| Variable | Treatment in the project | Limits of interpretation |
|---|---|---|
| Biometrics and voter registration | Record biometric coverage, registration reviews, cancellations and reinstatements by municipality and year, with explicit dates and missing values. | A voter may still be eligible to vote without biometric enrolment. Registration review and polling-day identification are distinct processes. |
| Second round for governor | Use a state-by-election indicator and consider its relationship with turnout and the competitiveness of the state contest. | The indicator is constant across municipalities in the same state in each election. In a single-state pilot, it may be indistinguishable from year effects. Estimate a separate coefficient only if it can be identified. |
| Weather, transport and access | Document extreme rainfall, closures, free transport and polling place changes in both rounds, using dated evidence. | Start with a checklist for individual cases. Include variables in the core model only if coverage and coding are comparable and were defined before testing. |
The variable inventory and data availability assessment must include these controls. Their use in numerical analysis depends on quality, variation and identification. Code missing data separately from the absence of an event. The TSE distinguishes biometric enrolment from registration reviews that may lead to cancellation. [D8]
Ecological inference as a supplementary module
The project will examine transition matrices using hierarchical R × C models where comparable units and sufficient information exist. Choose the R × C extension explicitly. Check whether a 2 × 2 implementation addresses the same modelling question before substituting it. [R6]
The matrices show model estimates of individual flows from aggregate data. Different matrices can reproduce the same margins. Report feasible accounting bounds, intervals and sensitivity to priors, covariates and assumptions, including cases where the information cannot support a precise estimate. This identification problem is fundamental to ecological inference. [R7]
Reconcile the electorates across rounds before fitting transitions. Even E₁ = E₂ can mask changes in the individual electors. Entries, exits and transfers require compatible categories or sensitivity analyses. With aggregate margins alone, the matrix may remain only partially identified. The module will be implemented in stages. The validity of the wider aggregate analysis will be assessed separately.
Four previous elections still provide a short time series. More municipalities do not create more independent state or national shocks. Historical data provide a benchmark. An undetectable mechanism may produce the same pattern as the benchmark, and the comparison cannot certify previous elections.
Distinguishing observations, accounting identities and estimates
Ecological inference remains an optional follow-up module. Aggregate deviation analysis can proceed when the data and calibration are adequate, even if that module is omitted, fails to converge or lacks sufficient identification.
- Observed. Counts and records present in the sources, subject to documentary validation.
- Derived from accounting identities. Net changes, rates and breakdowns that follow from the defined identities.
- Model estimates. Predictions, transition matrices and latent quantities, accompanied by assumptions and uncertainty.
Transition tables and charts must identify the third group explicitly. Avoid “actual flow”, “observed transfer” and “recovered true matrix” for cells inferred from aggregate margins alone. A Sankey diagram based on estimates must be clearly labelled and supplemented with a display of uncertainty. Use accounting waterfall charts where reliable estimates are unavailable.
10Comparable municipalities and external validation
Separate the characteristics used to assess comparability from the indicators that define an anomaly. Describe the initial municipality by size, electoral history, voter registration profile, socioeconomic context, geography, election administration and data quality.
Small initial set of covariates
Prioritise the log of the electorate, first-round composition, harmonised turnout and vote histories, age and education, and comparable measures of urbanisation and socioeconomic conditions. Add the operational inventory described in section 09. Reserve manually coded qualitative covariates, such as local endorsements, for specific hypotheses and justified extensions.
Use sources with compatible reference periods. Information must have been available before the election being predicted. For example, a prospective validation for 2014 cannot use 2022 data. Label retrospective contextual analysis separately.
Auditable similarity rule
- Define covariates, transformations and the treatment of missing values using the development data.
- Use Mahalanobis distance with regularised covariance and common support, or coarsened exact matching (CEM) stratification as an alternative analysis. Remove redundant dimensions.
- Set maximum distances (calipers) and comparison groups before examining held-out results. Report units without a suitable match.
- Check covariate balance and document the remaining differences.
- Compare flagged and unflagged cases within the same state. Then test the rule in comparable held-out states and document any revisions to the model.
A “multidimensional propensity score” will not be used to predict the anomaly classification itself and select controls on that basis. A conventional propensity score assumes a defined exposure. The initial aim here is descriptive similarity, which does not by itself identify a causal effect. Distance measures and calipers can be applied independently of a propensity score. [R8]
Pattern to replicate
Record the direction and size of deviations, each finalist's gain relative to O₁, gains relative to E₁, changes in turnout and invalid votes, predictive uncertainty, and concentration across polling places. The ΔF/O₁ ratios are dependent on the underlying counts and will be treated accordingly.
A reduction in abstention, holding the other components constant, increases the finalists' combined gains under the accounting identity. The identity cannot tell which candidate gains votes. A candidate's ratio above 100% is insufficient to classify the result as an anomaly.
11Protocol, false discoveries and robustness
Prospective protocol freeze and disclosure of prior analysis
Record the questions, rules, exclusions, primary indicators, candidate models, decision criteria, seeds and validation plan publicly or in a dated repository. List all results already examined and distinguish exploratory work from confirmation. Tests of the three cases already examined must be described as unblinded.
Claims of blinded assessment require genuinely held-out data and documented outcome access. Public availability does not establish blinding. Any change after examining a validation sample makes that round part of development and requires a new independent test.
Test families and false discovery control
- Proposed primary family. One two-sided joint test per eligible Rio de Janeiro municipality in 2026, with 92 hypotheses expected. The joint statistic must respect accounting dependencies. Reporting its components provides descriptive detail.
- Calibration. Produce valid p-values through out-of-sample prediction or simulation under the null model. Refit the pipeline where necessary and preserve dependence. Verify calibration before treating predictive tail probabilities or in-sample fitted residuals as uniform p-values.
- Default rule. Use a nominal FDR of 5% with Benjamini–Yekutieli (BY) and valid p-values to account for unknown dependence. Use Benjamini–Hochberg (BH) for sensitivity analysis, or as the primary rule only if its dependence assumptions are justified before the held-out test. [R9]
- Diagnostics and expansion. Register separate families for components, polling places and states. Selecting polling places after flagging requires selective or hierarchical adjustment, or a clear exploratory designation. The correction must account for how each subset was selected.
- Magnitude. Report counts, percentage points, effects per thousand electors and intervals. Set a substantive prioritisation threshold based on the study objectives and simulations before examining confirmatory results.
A 5% FDR limits the expected proportion of false rejections under the procedure's assumptions. It gives neither the probability of fraud in a municipality nor the exact error proportion in a particular list. Local FDR is an exploratory option given the 92 municipalities. Narrower confirmatory claims may require family-wise error control, such as Holm's procedure.
Validation and criteria for proceeding
- Temporal. Test a prediction for 2022 fitted on 2010–2018 data using temporally compatible variables. Use other forecast origins where feasible.
- Spatial. Validate on held-out spatial blocks and then in other states, assessing covariate support and institutional differences.
- Calibration and power. Measure interval coverage and false positives. Assess sensitivity in known synthetic scenarios. Sensitivity to actual fraud remains unverified.
- Model. Compare simple benchmarks, the hierarchical core model and the spatial extension. Check sensitivity to priors, overdispersion and shrinkage.
- Quality. Repeat using more reliable historical links and documented alternative rules for missing values, away-from-home voting and special polling sections.
- Scale. Compare municipality and polling-place results while avoiding pseudoreplication. Include unflagged controls to assess information lost through aggregation.
- Governance. Record the chosen configuration before opening each external test set and keep a version history of all changes.
If historical and simulation-based calibration is inadequate, the pilot will report descriptive results and diagnose limitations. Confirmatory identification of statistically unusual municipalities requires adequate calibration. BY and BH both require valid p-values and cannot fix model misspecification.
Empirical assessment of parsimony
Treat the lean model as an operational preference pending measurement. Define benchmarks and measure gains or losses from adding or removing groups of covariates.
- Compare the simple historical benchmark, lean core model and extensions on the same held-out temporal and spatial data.
- Assess calibration, interval coverage, predictive quality and stability of flagged cases. Quantify uncertainty in the comparisons.
- Measure sensitivity and false positives in explicitly defined synthetic scenarios. Report the scope of these scenarios separately from unknown real-world irregularities.
- Record data acquisition costs, computing time and harmonisation effort alongside the measured benefits.
- Preregister variable retention rules. Retain necessary controls unless the evidence supports removing them. An unverified claim of “90% of the power” is insufficient.
Report local coverage separately for municipalities flagged or unflagged at the aggregate level, eligible polling places in each group, places examined and places excluded on quality grounds. This table will show whether the analysis has inadvertently reintroduced the rejected municipal filter.
12Interpreting the findings
| Result | Permissible interpretation |
|---|---|
| Reconciled counts | The fields and totals are consistent with the verified identities. |
| Gain ratio above 100% | One candidate gains more votes than the other candidates received in the first round. The ratio alone does not measure improbability. |
| Statistical deviation | The observed result is unusual under the model and its assumptions, with quantified uncertainty. |
| Recurrence in comparable municipalities | The pattern may recur. Test this externally and compare with cases within the expected range. |
| Operational or territorial association | The relationship warrants investigation and may be affected by confounding. |
| Attribution to a software alteration | Requires evidence about the mechanism in addition to aggregate distributions. |
Technically supportable conclusion. “These units have statistically unusual results under the historical and contextual models tested, with the magnitude and uncertainty reported. The territorial analysis shows whether deviations are widespread or concentrated. External tests assess whether they recur in comparable municipalities.”
Improbability depends on the model. A deviation may reflect legitimate changes the model missed, poor model fit, data issues, administrative errors or irregularities. A hypothesis of software alteration requires additional evidence about the mechanism.
Aggregate results can reveal patterns and guide investigation. An alteration could produce a distribution indistinguishable from legitimate voting. Certification of process integrity requires more than the absence of an anomaly.
13Phased implementation and criteria for proceeding
| Stage | Deliverable | Criteria for proceeding |
|---|---|---|
| 0 · Revised design | v0.3, review matrix and decisions from the counter-analysis. | Completed in v0.3. |
| 1 · Protocol and data audit | Data dictionary by year, sources, inventory of results already examined and validation rules. | Reconcile coverage and populations. Freeze decisions for the confirmatory round. |
| 2 · Rio de Janeiro municipal core | All 92 municipalities, subject to eligibility, with historical series, indicators and a lean model. | Justify exclusions and document calibration and temporal validation. |
| 3 · Model comparison | Simple benchmarks, the hierarchical core model and a candidate spatial model. | Validation and sensitivity analyses must support the additional complexity. |
| 4 · Municipal assessment | Joint test, FDR, effect sizes, and profiles for cases and controls. | Confirmatory inference requires adequate calibration. |
| 5 · Polling places and ecological inference | Screen eligible polling places independently of municipal results. Investigate priorities and controls in depth. Produce optional matrices with bounds and uncertainty. | Ensure adequate coverage and assess selection. Report matrix precision only to the extent supported by the data. |
| 6 · Other states | Similarity rules and external tests with institutional controls. | Define the held-out dataset and criteria before examining its results. |
| 7 · Reproducible deliverables | Permitted data, code, container, manifest, technical report and visual communication. | Reproduction of the calculations and clear distinction between findings, hypotheses and limitations. |
The cost of each stage will be estimated after technical sampling of ingestion and harmonisation. Timing and budget remain to be estimated, and any gain in statistical power remains to be measured. The selection of less costly variables will be tested against benchmarks and alternatives.
Compare the core project with the historical benchmark, define spatial calibration and simulate known deviations. Record the confirmatory rules and any prior access to outcomes before evaluating 2026.
Additional v0.3 deliverables: A report on local coverage by municipality, an auditable table of A/B/C links, labels for observed, derived and estimated results, and an empirical comparison of the cost and performance of reduced and expanded specifications. The implementation plan calls for these deliverables.
14Assessment of the reviews and resulting decisions
The reviews complement each other. The Methodological Improvement Report details the techniques, while the Advisory Panel prioritises feasibility and communication. Each recommendation was assessed on its technical merits.
The project adopts most of the recommendations, with adjustments to reflect what aggregate data can identify. The matrix below records the review decisions. Testing remains to be done.
| Recommendation and source | Decision | Rationale and change |
|---|---|---|
| S01 · Initial scope Strategic review, pp. 1–2 | Accepted | Start with Rio de Janeiro's 92 municipalities. The core municipal analysis can proceed while nationwide harmonisation of polling-place data continues. |
| D01 · Primary sources Both reviews. Methodological review, p. 3 | Accepted | Require TSE data for inference. Retain secondary examples as clearly marked exploratory material and validate them before use. Protocol preparation can proceed in parallel. |
| G01 · Container and SHA-256 Strategic review, p. 1 | Accepted | Include instructions to reproduce the environment, a manifest and hashes. Use the hash to check file integrity. Establishing whether a recorded vote reflects the voter's intention requires separate evidence. |
| P01 · Blinded protocol Strategic review, pp. 1–2 | Adapted | Prospectively register rules and holdout samples and list outcomes already known. The three cases have already been examined. |
| P02 · Parsimony and T = 4 Strategic review, pp. 1–2 | Accepted | Use a reduced core specification, regularising priors and temporal and spatial validation. More municipalities provide more observations within each election. Independent elections supply a separate source of evidence. |
| P03 · More than 90% power Strategic review, p. 2 | Not adopted | The percentage lacks a demonstrated definition of power, a supporting experiment or a reference. The benefit of reducing covariates will be measured empirically. |
| M01 · Compositions Methodological review, p. 1 | Adapted | Model mutually exclusive categories over the same population. Signed gains require a separate specification from Dirichlet or log-ratio models. The centred log-ratio transformation retains a constraint and requires treatment of zeros. |
| M02 · Actual transition matrices via EI Both reviews, p. 1 | Adapted | Include R × C as a supplementary module. Estimate matrices consistent with the margins and assumptions and report bounds and uncertainty. The estimates describe possible transitions at the aggregate level. |
| M03 · Shrinkage Methodological review, p. 1. Strategic review, p. 2 | Accepted with limits | Use partial pooling for parameters, retain the raw data and test sensitivity. This can reduce noise and may also attenuate genuine signals. |
| M04 · FDR/BH/local FDR Methodological review, p. 1 | Adapted | Define the test family and set the FDR at 5%. Use BY as the conservative default when dependence conditions have yet to be demonstrated. Use BH when its assumptions hold and local FDR for exploration. Calibrated p-values are a prerequisite. |
| M05 · Mahalanobis/CEM/propensity scores Methodological review, p. 2 | Adapted | Formalise the distance measure, support and caliper. Prioritise regularised Mahalanobis distance or CEM. Propensity scoring requires a defined exposure, independent of the anomaly discovered during analysis. |
| M06 · W and BYM2 Methodological review, p. 2 | Accepted as a candidate | Specify the boundaries, neighbourhood structure, isolated units, priors and a non-spatial alternative. Separating regional patterns from irregularities causally would require evidence beyond BYM2. |
| C01 · Mandatory biometrics Both reviews. Methodological review, p. 2 | Adapted | Distinguish biometric coverage, register reviews and cancellations. Include them in the inventory and model them as data permit. Voters without biometric records may still be eligible to vote. |
| C02 · State-level runoff Both reviews. Methodological review, p. 2 | Accepted subject to identification | Include a state × year indicator and assess turnout. In a Rio de Janeiro-only analysis, the indicator may be confounded with election effects, leaving its coefficient unidentified. |
| C03 · Weather, transport and access Methodological review, p. 2 | Accepted | Use an evidence checklist for both rounds and add standardised covariates where coverage is comparable. |
| D02 · Voting away from home and absence declarations Methodological review, p. 2 | Adapted | Reconcile authorisations by office and round, origin and destination, and special categories. An absence declaration records an absence. Votes cast and turnout are counted separately. The cause of the differences E₂ − E₁ remains to be verified. |
| D03 · Geography guarantees linkage Methodological review, p. 3 | Adapted | Add coordinates and boundaries as evidence. Assess the strength of each link and leave unresolved cases open. Continuity of the electorate requires further evidence. |
| V01 · Waterfall or Sankey diagram Strategic review, p. 1 | Accepted with a choice of method | Use a reconciliation waterfall chart with observed values. Reserve Sankey diagrams for known flows or clearly labelled estimates. |
| S02 · Flagged units only Strategic review, p. 2 | Not adopted | Screen eligible polling places regardless of municipal flags. Prioritise only the detailed investigation and include controls. Version 0.3 documents coverage and exclusions and distinguishes screening from targeted investigation. |
| I01 · Falling abstention inflates Rᵢ Strategic review, p. 1 | Corrected | A fall in A contributes to the combined gains, all else being equal. The fall in A alone leaves each candidate’s gain undetermined. The denominator O₁ also matters. |
| I02 · Communication and evidence Both reviews | Accepted | Distinguish a ratio above 100%, a reconciled account and a statistical deviation. Identifying a software alteration requires separate evidence. |
v0.2 change log
- Defined the initial scope as the Rio de Janeiro municipal pilot, with stages and criteria for proceeding.
- Specified primary data governance, special populations and geographical harmonisation.
- Defined a count-based compositional core, partial pooling, BYM2 as a candidate and a supplementary R × C module with limitations.
- Detailed matching by distance, support and caliper, along with institutional and logistical controls.
- Defined the primary test family and FDR rule, and distinguished exploratory data already examined from holdout validation.
- Added reconciliation waterfall charts to the examples, normalised by the first-round electorate.
- Retained indicators for both finalists, both normalisations, the identities and the limits of interpretation.
Reviews provided by the proponent include the three-page Methodological Improvement Report and the two-page Advisory Panel review. Page references correspond to the submitted PDFs.
15Counter-analysis and design refinements
Document assessed: Contra análise dos pareceres do Proj v.01 A.Eleitoral, a one-page edition credited to Cláudio Stegun on 3 November 2026. The text compares the two reviews with v0.1 and identifies four extrapolations. [P3]
The project accepts all four points. They are consistent with the aim of detecting aggregate deviations within the limits of what those data reveal about individual behaviour. Version 0.2 incorporated all four qualifications. Version 0.3 makes them explicit in the criteria for selection, linkage, communication and validation.
| Point | Decision | Previous approach and refinement |
|---|---|---|
| CA01 · Municipal filter | Agreed | Version 0.2 allowed municipalities without flags, but its description of Phase II still focused on flagged cases and a sample of controls. Version 0.3 distinguishes screening of all eligible polling places, historical comparison with assessed links and targeted investigation. |
| CA02 · Actual flows | Agreed | Version 0.2 already acknowledged that the data cannot identify individual flows and treated R × C as supplementary. Version 0.3 labels observations, derivations and estimates and allows aggregate analysis to proceed without a transition matrix. |
| CA03 · Geography as a guarantee | Agreed | Version 0.2 already treated coordinates as supporting evidence. Version 0.3 puts classes A/B/C into practice, separates primary use from sensitivity analysis and requires a report on coverage and exclusions. |
| CA04 · More than 90% of power | Agreed | Version 0.2 already rejected this claim. Version 0.3 requires comparison of specifications on the same holdout sets, using calibration, prediction, simulated sensitivity and cost metrics. |
Scope of the critiques
The reviews proposed several approaches the project retains. Ecological models can produce conditional estimates, coordinates can support harmonisation, simpler models can reduce costs, and flagged municipalities can receive priority for investigation. Selection remains open to other municipalities. Individual flows, continuity of the electorate and quantitative performance each require supporting evidence.
The project retains its phased implementation and the pilot covering Rio de Janeiro's 92 municipalities. The counter-analysis evaluates the methodology. The revised design retains empirical evaluation and data checks as separate stages.
v0.3 change log
- Polling-place screening independent of municipal results, with documented eligibility and coverage.
- Distinction between cross-sectional screening, historical comparison and targeted investigation.
- A/B/C classes for historical links and handling of unlinked units.
- Labelling of observed, derived and estimated results, with ecological inference explicitly optional.
- Protocol for empirically comparing simpler and more complex specifications, leaving quantitative performance to be measured.
- Retained the numerical examples, formulae, FDR, controls and references from v0.2.
The refinements above are project specifications. Their implementation and validation remain pending.
16Sources and references
Sources consulted during project design. External links require an internet connection. The document's content, navigation and formulae work offline.
- [D1] TSE — Election results. Portal for accessing the historical series.
tse.jus.br/eleicoes/resultados-eleicoes - [D2] TSE — 2010 results. Dataset that includes votes by polling section.
dadosabertos.tse.jus.br/dataset/resultados-2010 - [D3] TSE — 2010 electorate. Includes electorate data by polling place.
dadosabertos.tse.jus.br/dataset/eleitorado-2010 - [D4] TSE — 2014 electorate.
dadosabertos.tse.jus.br/dataset/eleitorado-2014 - [D5] TSE — 2018 electorate.
dadosabertos.tse.jus.br/dataset/eleitorado-2018 - [D6] TSE — 2022 electorate. Includes polling places, temporary transfers and combined polling sections.
dadosabertos.tse.jus.br/dataset/eleitorado-2022 - [R1] Zhang, M.; Alvarez, R. M.; Levin, I. (2019). Election forensics: Using machine learning and synthetic data for possible election anomaly detection. PLOS ONE, 14(10), e0223950.
doi.org/10.1371/journal.pone.0223950
Supports combining historical data, contextual data and detection models. The Brazilian specification in this document is a working proposal. - [R2] The Modifiable Areal Unit Problem in Political Science (2025). Political Analysis.
doi.org/10.1017/pan.2025.2
Discusses how the scale and boundaries of aggregated units affect results and inferences. - [R3] Montgomery, J. M.; Olivella, S.; Potter, J. D.; Crisp, B. F. (2015). An Informed Forensics Approach to Detecting Vote Irregularities. Political Analysis, 23(4), 488–505.
doi.org/10.1093/pan/mpv023 - [D7] TRE-RJ — 2022 elections. Institutional record of the 92 municipalities and election administration.
tre-rj.jus.br/institucional/memoria/eleicoes-2022 - [D8] Electoral Justice — Biometrics. Distinguishes biometric identification from a summons for a voter registration review.
justicaeleitoral.jus.br/biometria/ - [D9] TSE — Service charter. Sections on voting away from home and declaring an absence.
Voter service charter - [D10] TSE — Declaring an absence from voting.
Declaring an absence from voting - [R4] Tsagris, M.; Alenazi, A.; Stewart, C. (2023). Flexible non-parametric regression models for compositional response data with zeros. Statistics and Computing.
doi.org/10.1007/s11222-023-10277-5
Reference for transformation constraints and treatment of zeros. The proposed count family is a design choice for this project. - [R5] Stan — Spatial Models: Intrinsic Auto-Regressive Models for Areal Data. Technical documentation for ICAR and BYM2, with original references.
mc-stan.org/learn-stan/case-studies/icar_stan.html - [R6] Rosen, O.; Jiang, W.; King, G.; Tanner, M. (2001). Bayesian and Frequentist Inference for Ecological Inference: The RxC Case. Statistica Neerlandica, 55, 134–156.
Article page on Gary King's website - [R7] Wakefield, J. (2004). Ecological Inference for 2 × 2 Tables. Journal of the Royal Statistical Society: Series A, 167(3), 385–425.
doi.org/10.1111/j.1467-985x.2004.02046_1.x
Discusses non-identifiability from aggregate margins. - [R8] MatchIt — official documentation. Distances, calipers, support and balance diagnostics.
kosukeimai.github.io/MatchIt/reference/matchit.html - [R9] Benjamini, Y.; Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of Statistics, 29(4), 1165–1188.
doi.org/10.1214/aos/1013699998
BH requires certain dependence conditions. BY offers a conservative modification for other dependence structures, given valid p-values. See also the official R documentation. - [P3] Contra análise dos pareceres do Proj v.01 A.Eleitoral. One-page document submitted by the proponent, with the 3 November 2026 edition credited to Cláudio Stegun.
Counter-analysis supplied to the project.
Origin of this version: This version consolidates the decisions and calculations and reviews the two reports and counter-analysis received on 3 November 2026. It sets out the proposed procedures for analyses that remain to be carried out.
17First empirical run and pending proposal
Reconciled municipal panel
The pilot integrated ten TSE files for 2010, 2014, 2018, 2022 and 2026 and recorded them in a SHA-256 manifest. The panel contains 920 municipality–round records, representing 460 pairs across rounds. The detailed count and votes by candidate matched across 2,148 electoral-zone keys. All accounting residuals checked were zero, including in the reconciliation of finalists' gains.
For 2010 and 2014, the pilot used votes by candidate because there was no separate field for valid votes by candidate. Their sum was checked against the official total for each electoral-zone key. The panel retains the published population of eligible voters and separates records marked as votes cast away from home. In 2026, the TSE reported 39 duplicate origin links nationwide. Any local adjustment requires documentary evidence of how those links are distributed across municipalities. TSE, 2026 Results.
Exploratory 2022 temporal test
The benchmark predicts the change between rounds per 100 first-round eligible voters using the municipal median for 2010, 2014 and 2018. For finalists' gains, it aligns their positions by their first-round vote totals in each municipality. The intervals are symmetric and use the 90th percentile of the absolute error when leaving out one historical election at a time.
| Indicator | Mean absolute error per 100 eligible voters | 2022 municipalities within the interval |
|---|---|---|
| Abstention | 3.19 | 34.8% |
| Blank votes | 0.40 | 97.8% |
| Invalid votes | 1.14 | 100.0% |
| Valid votes | 3.22 | 26.1% |
| Gain of the finalist who led in the first round | 5.15 | 88.0% |
| Gain of the other finalist | 7.54 | 23.9% |
Several indicators have insufficient coverage to use these intervals as a flagging rule. This stage accessed the 2022 errors and the 2026 municipal data. Any adjustments informed by the 2022 errors make that round part of model development. The protocol must identify genuine holdout sets and record access to outcomes.
Mahlmeister · awaiting approval
The proposal remains open for a future decision. Reproduction will take place in a separate session. If included in the pilot, the evaluation will distinguish the model that uses historical and first-round data from the one that learns from partial results of the second round itself. Municipal and polling-place performance should be evaluated in geographical holdout blocks. The historical benchmark in this pilot was calculated separately.