MCPS — Mexico City Prospective Study allele frequencies

MCPS (Mexico City Prospective Study) is a large cohort with mixed whole-genome, whole-exome, and array-based genotyping, contributing population-specific allele frequencies with particular strength in Mexican/Latino ancestry — a population under-represented in gnomAD and most other public frequency panels. iris reads it as 23 bgzipped, tabix-indexed, per-chromosome TSVs (chr{N}.freq.cpra.tsv.bgz) at /mnt/cephfs/hot_nvme/mcps/mcps-variant-browser-afs/ (5.1 GB total) — the opposite layout from dbSNP and CADD's single whole-genome files.

This page documents the files and shows how iris reads them via McpsAFAnnotator (src/iris/adapters/driven/annotators/mcps.py). Every number and example below ran against the real files.

Scale

22 autosomes + X — no Y, no MT. Full genome-wide scan, real counts:

Contig	Rows
chr1	10,916,583
chr21	1,720,825
chrX	6,464,980
Total (all 23 files)	141,802,412

(Full scan of all 23 files: 84.9s.)

Schema

#CHROM  START  END  REF  ALT  ID  CPRA  SOURCE  STON  GHOM  AHOM
AN_RAW  AC_RAW  AF_RAW  AN_AFR  AC_AFR  AF_AFR  AN_EUR  AC_EUR  AF_EUR  AN_MEX  AC_MEX  AF_MEX

The file ships its own header line, but TabixReader._query() strips it on random-access fetches — mcps.toml [fieldnames] declares the same 23 names explicitly so csv.DictReader can reconstruct rows regardless.

Populations

Four population buckets, each with its own AN/AC/AF triplet, mapped to AlleleFrequency.population via mcps.toml [[populations]]:

Column prefix	iris population label	Meaning
*_RAW	mcps	Whole cohort, all ancestries
*_AFR	mcps_afr	African-ancestry subset
*_EUR	mcps_eur	European-ancestry subset
*_MEX	mcps_mex	Mexican/Amerindian-ancestry subset — MCPS' distinguishing contribution

McpsAFAnnotator only emits an AlleleFrequency for a population when at least one of AF/AC/AN is present (not .); populations with no data for a given variant are simply absent from the result, not zero-filled.

SOURCE — genotyping platform per variant

Five distinct values observed across a full scan of all 23 files:

WGSOnly, WESOnly, ArrayOnly, WESOverlap, ArrayOverlap

McpsAFAnnotator(freq_dir=..., source=["WGSOnly"]) restricts matching to rows whose SOURCE is in that list — useful to exclude array-only variants (typically pre-selected common SNPs, not a representative random sample) when computing rarity statistics.

STON / GHOM / AHOM — passed through as metadata, not formally documented

These three columns aren't parsed into any typed domain field — they're copied verbatim into VariantAnnotation.metadata.data (keys ston, ghom, ahom, alongside mcps_id from the ID column) via mcps.toml [metadata_int_fields]/[metadata_str_fields]. Their exact denotation isn't documented in the source files themselves; by column-naming convention they likely mean singleton count/flag, genotype-homozygous count, and allele-homozygous count respectively, but treat that as an informed guess, not a verified definition — iris treats them as opaque pass-through data for exactly this reason, rather than mapping them into typed fields it can't vouch for.

AN/AC are fractional, not integer sample counts

Real row (21:5033276 C>T, live-annotated below): AN_EUR=6046.505, AN_MEX=13084.461 — not whole numbers. This is a weighted/effective allele count (likely accounting for relatedness or imputation dosage across the mixed WGS/WES/array sources), not a literal sample tally. AlleleFrequency.count/.total are typed int | None, so McpsAFAnnotator truncates via int(float(x)) — 6046.505 becomes 6046, silently dropping the fractional part. AF itself (a float) is unaffected and carries full precision.

Bug found and fixed while writing this page: chrX never matched

chrX.freq.cpra.tsv.bgz's internal tabix contig name is "23", not "X" or "chrX":

>>> pysam.TabixFile(".../chrX.freq.cpra.tsv.bgz").contigs
['23']

Verified against every one of the file's 6,464,980 rows (full scan, not a sample): CHROM and CPRA — the two structured, position-bearing columns — are "23" on 100% of rows, with zero exceptions. ID, by contrast, is a free-text identifier column (like a VCF ID field) and is not uniformly "23" or "X": 96.68% start with X: (a synthesized CPRA-style placeholder), 3.05% start with 23: (same placeholder, inconsistent prefix choice), and 0.28% are external identifiers with no chromosome prefix at all — rsIDs and genotyping-array probe names (rs5939320, JHU_X.2701482, …), all correlating exactly with SOURCE=ArrayOnly/ ArrayOverlap rows, which naturally carry their platform's own ID instead of a synthetic one. None of this matters for the bug or its fix: iris never reads ID, only CHROM/CPRA, and those are unconditionally "23".

This broke matching in two independent places:

1. TabixReader._query()'s chromosome resolution (resolve_chromosome()) only tries the bare name and a chr-prefix toggle — neither bridges "X" → "23" — so the tabix fetch itself never found the file's rows.
2. Even with the fetch fixed, _cpra_to_key() parses the row's own CPRA column, which would still yield chrom "23" — not matching a caller's "X"-keyed targets.

Both are now fixed in McpsAFAnnotator, verified live:

annotator = McpsAFAnnotator(freq_dir=MCPS_DIR)
annotator.annotate([va("X", 10_009, "A", "G")])[0].allele_frequencies
# 4 AlleleFrequency entries (mcps/mcps_afr/mcps_eur/mcps_mex) — was ()

annotator.query_region(GenomicPosition(chromosome="X", start=10_000, end=10_020))
# 6 VariantAnnotation, all with position.chromosome == "X" — was []

The fix has two parts, matching the two failure points above:

1. Tabix fetch: resolve_file_contig() (src/iris/adapters/driven/_helpers.py, shared with dbNSFP's analogous MT bug) falls back to a file's sole contig when the standard bare/chr-toggle resolution fails — safe here (unlike making the generic resolve_chromosome() itself do this) specifically because McpsAFAnnotator only ever opens path after already selecting it for the exact chromosome being queried via file_pattern, so a lone differently-named contig can't be some other, genuinely unrelated chromosome.
2. Row parsing: _load_chrom()/_row_to_annotation() now override whatever chromosome the row's own CPRA column carries with the caller's already-known canonical chromosome, rather than trusting CPRA's literal "23" directly (harmless for this fix specifically, since CPRA is 100% consistent — see above — but the override also means iris never has to special-case "23" anywhere outside this one translation step).

This is the same root-cause shape as dbSNP's RefSeq contig handling — a contig name that generic chr-prefix-only resolution can't bridge — solved the same way dbSNP solves it: locally, in the annotator that already knows which chromosome a given file represents, rather than in the shared, necessarily-conservative resolve_chromosome().

Access patterns from iris

Registering the source

Already registered in this environment's ~/.iris/config.toml:

[sources.mcps]
host = "local"
path = "/mnt/cephfs/hot_nvme/mcps/mcps-variant-browser-afs"

from iris.adapters.driven.annotators.mcps import McpsAFAnnotator

annotator = McpsAFAnnotator.from_registry()                        # all sources
annotator = McpsAFAnnotator.from_registry(workers=4)
annotator = McpsAFAnnotator(freq_dir=MCPS_DIR, source=["WGSOnly"])  # direct construction, source-filtered

annotate() — point-lookup enrichment

Live run against a real row with non-trivial metadata:

result = annotator.annotate([va("21", 5_033_276, "C", "T")])

AlleleFrequency(frequency=5.03e-05, count=1, total=19896, population='mcps',      allele='T')
AlleleFrequency(frequency=0.0,      count=0, total=765,   population='mcps_afr',  allele='T')
AlleleFrequency(frequency=0.0,      count=0, total=6046,  population='mcps_eur',  allele='T')
AlleleFrequency(frequency=7.64e-05, count=1, total=13084, population='mcps_mex',  allele='T')
metadata: Metadata(source='mcps', data={'ston': 1, 'ghom': 0, 'ahom': 1, 'mcps_id': '21:5033276:C:T'})

(total values above are the truncated AN_* — the untruncated source was 19896, 765.032, 6046.505, 13084.461.)

query_region() — bulk region scan

Unlike annotate() (point lookup against known keys), query_region() builds fresh VariantAnnotation objects for every row overlapping a region — the same shape as RgcmeAFAnnotator.query_region() and uvar's fetch(). Subject to the same chrX limitation above (region resolution goes through the same resolve_chromosome() call).

workers — chromosome-partitioned parallel fetch

Parallelizes across files, not within one — workers controls how many of the (up to 23) per-chromosome files are opened and queried concurrently, each via its own TabixFile handle. Given each file is small (largest is ~150 MB, vs. dbSNP/CADD's single 75-90 GB files), the win from workers here comes from overlapping I/O across many files in a multi-chromosome query, not from splitting one huge file.

Design notes & caveats

• chrX lookups used to silently return nothing — fixed; see above.
• No Y, no MT coverage at all — 23 files, only autosomes + X (real absence, not a bug — there's no chrY.freq.cpra.tsv.bgz/chrMT.* file to fix a lookup for).
• AN/AC are fractional and get truncated to int — see above. Use AF directly rather than reconstructing it from truncated count/total if precision matters.
• STON/GHOM/AHOM are opaque pass-through metadata — not mapped to typed fields; see above.
• Per-chromosome files, unlike dbSNP/CADD — workers here parallelizes across files, not within a single large one.
• SOURCE filtering matters for rarity analysis — ArrayOnly/ ArrayOverlap rows reflect a pre-selected genotyping array content, not a random population sample; consider source=["WGSOnly", "WESOnly", "WESOverlap"] when that bias matters.

Reference

iris.adapters.driven.annotators.mcps

MCPS allele frequency annotator via tabix random-access.

McpsAFAnnotator

Bases: TabixVariantAnnotator

Annotates variants with MCPS allele frequencies (global + AFR/EUR/MEX ancestries).

Expects bgzipped, tabix-indexed per-chromosome files named chr{N}.freq.cpra.tsv.bgz under freq_dir. Column layout is defined in mcps.toml and matches the file header::

#CHROM  START  END  REF  ALT  ID  CPRA  SOURCE  STON  GHOM  AHOM
AN_RAW  AC_RAW  AF_RAW  AN_AFR  AC_AFR  AF_AFR
AN_EUR  AC_EUR  AF_EUR  AN_MEX  AC_MEX  AF_MEX

workers class-attribute instance-attribute

workers = 1

Default parallel tabix workers, used when annotate()/query_region() omit an override.

from_registry classmethod

from_registry(
    source_name="mcps", registry=None, *, workers=1
)

Build a McpsAFAnnotator resolving source_name from the iris DataRegistry.

Source code in src/iris/adapters/driven/annotators/mcps.py

@classmethod
def from_registry(
    cls, source_name: str = "mcps", registry=None, *, workers: int = 1
) -> "McpsAFAnnotator":
    """Build a McpsAFAnnotator resolving *source_name* from the iris DataRegistry."""
    from iris.config import get_registry

    r = registry or get_registry()
    return cls(freq_dir=r.resolve_path(source_name), workers=workers)

annotate

annotate(items, *, workers=None)

Annotate variants with MCPS allele frequencies.

PARAMETER	DESCRIPTION
items	Variants to annotate. TYPE: Sequence[VariantAnnotation]
workers	Parallel tabix workers for this call — controls how many per-chromosome files are queried in parallel, each opening its own TabixFile. Defaults to self.workers (set at construction) when omitted. TYPE: int | None DEFAULT: None

Source code in src/iris/adapters/driven/annotators/mcps.py

def annotate(
    self, items: Sequence[VariantAnnotation], *, workers: int | None = None
) -> list[VariantAnnotation]:
    """Annotate variants with MCPS allele frequencies.

    Args:
        items: Variants to annotate.
        workers: Parallel tabix workers for this call — controls how many
            per-chromosome files are queried in parallel, each opening its
            own ``TabixFile``. Defaults to ``self.workers`` (set at
            construction) when omitted.
    """
    targets: set[_Key] = set()
    for va in items:
        maybe_key = va.variant.as_key()
        if maybe_key is None:
            continue
        targets.add(maybe_key)
    if not targets:
        return list(items)
    chroms = {key[0] for key in targets}
    n_workers = self.workers if workers is None else workers
    freq_map = self._load(chroms, targets, workers=n_workers)
    return [self._enrich(va, freq_map) for va in items]

query_region

query_region(regions, *, workers=None)

Return all MCPS variants whose positions overlap the given regions.

Each row in the per-chromosome frequency file becomes a fresh VariantAnnotation containing the parsed allele frequencies. workers controls how many chromosome files are scanned in parallel; defaults to self.workers when omitted. Files that do not exist on disk are silently skipped.

Source code in src/iris/adapters/driven/annotators/mcps.py

def query_region(
    self, regions: GenomicPosition | Sequence[GenomicPosition], *, workers: int | None = None
) -> list[VariantAnnotation]:
    """Return all MCPS variants whose positions overlap the given regions.

    Each row in the per-chromosome frequency file becomes a fresh
    ``VariantAnnotation`` containing the parsed allele frequencies.
    ``workers`` controls how many chromosome files are scanned in
    parallel; defaults to ``self.workers`` when omitted. Files that do
    not exist on disk are silently skipped.
    """
    if isinstance(regions, GenomicPosition):
        regions = (regions,)

    by_chrom: dict[str, list[GenomicPosition]] = {}
    for region in regions:
        by_chrom.setdefault(region.chromosome, []).append(region)

    batches = [
        (Path(self.freq_dir) / self.file_pattern.format(chrom=chrom), chrom_regions)
        for chrom, chrom_regions in by_chrom.items()
        if (Path(self.freq_dir) / self.file_pattern.format(chrom=chrom)).exists()
    ]
    if not batches:
        return []

    n_workers = min(self.workers if workers is None else workers, len(batches))
    if n_workers <= 1:
        results: list[VariantAnnotation] = []
        for path, chrom_regions in batches:
            results.extend(self._scan_region(path, chrom_regions))
        return results

    results = []
    with ThreadPoolExecutor(max_workers=n_workers) as pool:
        futures = [pool.submit(self._scan_region, path, cr) for path, cr in batches]
        for future in as_completed(futures):
            results.extend(future.result())
    return results