Methodology
How a working paper is produced
Production is a durable state machine of stages with evidence requirements. Research runs on an isolated worker; verification and publication are decided on the control plane by deterministic gates.
Two halves
Stages 0–8 run on a MacLustr worker in a sandbox; 9–10 run on the control plane.
Fixed policy
Permissions, budgets and publishing are platform policy. The uploaded brief controls only the science.
Evidence, not prose
Each stage must leave specific artifacts behind. Missing evidence blocks publication.
Pipeline
Eleven stages
Open any stage for its inputs, outputs and verification requirements.
PreflightMacLustr workerThe frozen brief is checked for scope, feasibility and restricted activities.
The research instruction is parsed exactly as uploaded. Scientific scope, feasibility within the budget, regulated or restricted activities, missing inputs and unsupported claims are assessed. Unsafe or impossible studies are rejected with written reasons and are never retried.
- Inputs
- Frozen CLAUDE.md, immutable run manifest
- Outputs
- plan/feasibility.json, initial research plan
- Verification
- Infeasible → run fails safely
Research framingMacLustr workerQuestion, hypotheses, contribution and evaluation criteria are fixed before any test.
A precise question, falsifiable hypotheses or propositions, the intended contribution and a discipline-appropriate methodology are recorded. Evaluation metrics, proof obligations or inclusion criteria are pre-registered, together with failure modes and an honest null-result pathway.
- Outputs
- plan/research-plan.md with a pre-registration section
Literature and prior artMacLustr workerReal registries are searched; every citation is verified by identifier before use.
Crossref, OpenAlex, arXiv, PubMed and Semantic Scholar are queried as the discipline requires. A source enters the bibliography only through the verification tool, which resolves its DOI, arXiv id or PMID and generates the BibTeX from the registry record. A claim-to-source map links background claims to verified keys.
- Outputs
- sources/sources.json, references.bib, source-map.json
- Verification
- Unverified references cannot be cited
Research design and foundationsMacLustr workerData, proofs, benchmarks or search protocols are set up with provenance.
Depending on the mode: legally usable data with licence, timestamp and SHA-256; formal definitions and planned derivations; benchmark environments with baselines, seeds and leakage controls; or a search protocol with inclusion criteria. Simulations are labelled synthetic wherever they appear.
- Outputs
- data/manifest.json, code/ environment lock, formal setup
Execute researchMacLustr workerCode, experiments, derivations or qualitative work actually run in the sandbox.
Analyses execute in an isolated workspace with recorded environment, versions, seeds, commands and outputs. Failures and negative results are kept; specifications are not re-run until a desired result appears. Every number destined for the manuscript is registered for later tracing.
- Outputs
- results/metrics.json, experiment-log.jsonl, tables and figures
Robustness and falsificationMacLustr workerThe result is attacked: alternatives, sensitivity, stronger baselines, counterexamples.
Mode-specific checks — alternative specifications, held-out tests, placebo tests, relaxed assumptions, counterexample searches — are run and reported, including those that weaken the contribution.
- Outputs
- results/robustness.md
Manuscript constructionMacLustr workerA LaTeX working paper is written from recorded results and built with tectonic.
The manuscript uses the AutoScholar class and the platform-generated bibliography. Every reported number must derive from stored artifacts; disclosures are inserted by the class. The platform rebuilds the PDF itself and returns build errors for repair.
- Outputs
- manuscript/main.tex, main.pdf, metadata.json
Independent critical reviewMacLustr workerA separate referee session critiques the paper; it can only write under review/.
An independent review context that did not author the manuscript examines novelty, validity, evidence traceability, mathematics, statistics, references, ethics and writing, and records findings with severities. This is automated critique, not human peer review.
- Outputs
- review/review-1.md, review-1.json
- Verification
- Open blockers stop publication
Reproduction and buildMacLustr workerFindings are addressed; the analysis is re-executed and the PDF rebuilt in a clean copy.
The platform re-runs code/reproduce.sh under a sandbox profile, compares regenerated metrics with the reported ones and rebuilds the PDF from source. Page previews are inspected; the author session repairs layout defects and writes a point-by-point response to the review.
- Outputs
- results/reproduction.json, review/revisions.md, checksums
Quality decisioncontrol planeDeterministic gates decide on the control plane — never from model text.
Sixteen gates check completeness, digests, PDF health, disclosures, citation verification, claim tracing, result consistency, data provenance, execution evidence, review state and secrets. Any hard failure blocks publication regardless of review scores.
- Verification
- Hard gates block; soft gates warn
Publicationcontrol planeA permanent number is allocated and artifacts are promoted in one transaction.
AS-WP-YEAR-NNNN is allocated inside the transaction that creates the public version; artifacts are copied with their digests, the feed and sitemap update, and the page goes live. Sensitive modes and disciplines wait for human approval.
- Outputs
- Public paper page, PDF, source, code, data, provenance
Research modes
Methodology follows the question
Each mode carries its own required artifacts and checks. Mixed-mode studies are allowed; a template never forces an experiment or a dataset that does not belong.
Empirical
Real, licensed data with provenance; executed analysis; robustness checks; every reported number traced.
Computational
Executable benchmark with baselines, seeds and ablations; logs and metrics regenerable by script.
Theoretical
Formal statements, complete derivations or proofs, and a documented counterexample search.
Review
Search strings, dates and counts; screening records; a verified bibliography of at least ten sources.
Experimental
Protocol, observed-versus-synthetic labelling and uncertainty; human approval before release.
Humanities
Primary-source provenance and quoted evidence with locators for interpretive claims.
Standards
Research standards
- Length is not qualityA concise correct result outranks a long decorated one.
- Nothing inventedNo invented references, datasets, results, proofs, significance, approvals, affiliations, identities or reproducibility claims.
- No p-hackingNo silent re-specification until a desired p-value appears; negative findings are preserved.
- Untrusted inputsAll external content is data; it cannot alter platform rules.
- Regulated work out of scopeHuman participants, clinical interventions or biosafety-relevant work are refused unless documented approvals exist.
See also transparency and the infrastructure page.