USGS MRDS Imbalance & Earth MRI: REE Model Leaderboards Mislead

High-scoring cells on national REE prospectivity maps overlap USGS MRDS occurrences and their immediate buffers, judging by public model outputs and Earth MRI evidential layers. That overlap means the classifier has learned the positions of old mines, not the physical signals of rare-earth enrichment.

The root cause is uniform random negative sampling. With many MRDS records but only a small fraction tagged for REE commodities, standard practice scatters background points across every map cell. Those negatives force the model to separate targets by global distance to known deposits, producing a smooth mine-proximity halo. The edge in REE prospectivity is not a new neural network; it is the decision to reject that sampling assumption.

The practical fix is to generate negatives from geological evidence—lithology, alteration, geophysics—rather than uniformly across the map. In models that make this switch, the mine-buffer contribution drops, exposing the leaderboard inflation. The most-touted maps were not wrong about geochemistry; they were wrong about sampling, and the MRDS imbalance turned a benchmark into a mirror of legacy mining.

The Mechanism

USGS MRDS is the label source, and its imbalance dictates every design choice downstream. The database holds many mineral occurrence records, but only a small fraction carry REE commodities. On that distribution, accuracy is a trap: a model that labels every cell "not REE" scores trivially correct while discovering nothing. That is why the ensemble's first mechanic is heavy class weighting and precision-recall AUC. Precision-recall AUC rewards the model for retrieving the rare REE cells instead of congratulating it for ignoring them, and the heavy weighting forces the classifier to treat a missed carbonatite as far costlier than a false alarm on barren ground.

Every target is a cell. Each cell's feature vector is extracted from USGS Earth MRI airborne surveys: the potassium, eTh, and eU radiometric channels, plus reduced-to-pole aeromagnetics. The cell is the atomic unit — probabilities are assigned per cell, and the decision rule spends field dollars per cell, so everything downstream inherits this grid. Finer resolution would be noise; coarser would blend the radiometric contrast that carries the signal.

The discriminative signal is the eTh/eU ratio. Carbonatite-hosted REE deposits in the training data show elevated ratios, while barren shale and basalt sit below that boundary. Random Forest and XGBoost use this ratio as a primary split, which is why the ensemble can separate ore-hosting carbonatite from its most common barren wall rocks using radiometrics alone — before any geochemistry is collected. LASSO-logistic cannot threshold the ratio as cleanly, which is precisely why it is a useful veto rather than the lead voter.

The negative-sample mechanic is where most replications go wrong. Random background points inherit MRDS's exploration bias: records cluster near old mines, so a random negative set is really a set of "near-mine" cells. Density-controlled background extraction thins cells in inverse proportion to local record density — over-explored districts yield fewer background cells, while unexplored ground yields more. That correction, applied at sampling time, is the difference between a model that maps geology and a model that maps mining history. It is also the difference behind the ensemble's AUC gain: the density-controlled version consistently beats the random-negative version by the margin the thesis claims, because the model finally sees what barren rock actually looks like away from historic workings.

The vote mechanic is deliberately conservative. Each of the three models outputs a probability per cell; an ensemble target requires at least two of the three to exceed the calibrated cutoff. Single-model false positives skew REE exploration badly — one tree model can lock onto a single anomalous radiometric channel and illuminate a region the other two models read as barren. The 2-of-3 vote suppresses those lone-wolf calls at the cost of some recall, which is the right trade when every field dollar counts.

The myth to kill here is that stacking more geophysics automatically improves the model. In a Basin and Range test, adding full-tensor gravity dropped the ensemble's AUC because basin-thickness noise overwhelmed the radiometric thorium signal. The mechanism does not want more data; it wants the right data. The Earth MRI radiometrics are sufficient, and thick-basin gravity introduces a confounder that the tree models happily overfit.

The Evidence

USGS Earth MRI has flown extensive airborne geophysics, and every radiometric and magnetic grid used in this guide is a public-domain download from the USGS data portal. That reproducibility is the foundation of the approach: the ensemble runs on the same open grids any exploration team can pull today, not on vendor-locked survey data.

The quantitative core is a Basin and Range benchmark (Briggs, Stanford), which trained Random Forest, XGBoost, and LASSO-logistic regression on identical USGS MRDS labels and Earth MRI features. The result is the sharpest single argument for the 2-of-3 rule: the ensemble reaches a higher AUC than the best single model, XGBoost.

That gain only has meaning because the negatives were density-controlled. According to Zuo & Wang (Ore Geology Reviews), random negative sampling inflated reported AUC across mineral prospectivity studies, and especially in REE-specific maps. A model validated against random negatives carries a hidden performance tax, and it is highest for exactly the REE subclass this guide is built to find.

One feature does heavy lifting inside all three models. According to Verplanck et al. (USGS Critical Mineral Resources project), many REE occurrences in MRDS sit near a carbonatite or alkaline intrusive complex. That is the quantitative basis for the distance-to-carbonatite feature — a spatial prior that encodes the dominant REE deposit model in the Basin and Range.

The guide's rejection of weights-of-evidence is also a measured result, not a stylistic preference. A Stanford MSc thesis benchmark (department repository) showed Random Forest improved area-under-precision-recall over weights-of-evidence on identical MRDS features. For a sparse positive class like REE occurrences, AUPRC is the stricter test — and weights-of-evidence failed it.

The same benchmark kills the assumption that more geophysics is always better. Adding full-tensor gravity dropped the ensemble's AUC — worse than XGBoost alone — because basin-thickness noise overwhelmed the radiometric thorium signal. The feature-selection lesson: the Earth MRI thorium channel is worth more than a gravity tensor where basin sediment masks the basement.

Before spending field dollars, ask for three things: the per-model AUC table, the negative-sampling protocol, and the 2-of-3 agreement map at the probability cutoff. Any single-model AUC reported without density control is not a reliable basis for drilling a carbonatite.

Decision Framework

The single-model leaderboard is a trap. XGBoost posts the best solo precision at high recall, but it is also the most fragile vote on the panel: under a mislabeled-negative attack, its AUC drops, while Random Forest — at larger memory — loses less. Neither alone should authorize field exploration. The decision unit for the field season is the 2-of-3 agreement ensemble. The comparison below ranks the single models plus that ensemble across the criteria that matter when the deliverable is a drill budget rather than a conference poster.

XGBoost is the explicit single-model winner: highest precision at high recall, moderate training time, and standard tree-ensemble settings. But that precision is brittle. Missing REE labels in USGS MRDS are the rule rather than the exception, so a mislabeled-negative attack is a realistic stress test — and XGBoost loses AUC under it. Random Forest is the stability runner-up: it uses more memory per test grid but loses less AUC under identical noise. That asymmetry is what makes RF the tie-breaker in contested districts: when the other two votes split, the model that resists label noise should cast the deciding vote.

The ensemble row wins the table for field decisions. Requiring at least two of three models to exceed the probability cutoff on the same cell raises precision at high recall and cuts false positives on the Basin and Range benchmark. That is the difference between a ranking exercise and a defensible drill budget.

The cheap-start path keeps compute honest for two-person junior teams: run LASSO-logistic first — quickly on a laptop — to post a provisional top-ranked map, then deploy Random Forest and XGBoost only on those cells. A cell that LASSO filters out has already cast its no vote, so the 2-of-3 structure survives the screening step. This reduces compute on the grid.

The resulting decision tree, applied cell by cell:

What the Data Doesn't Tell You

The reported AUC carries a hidden tax. When entire REE districts — not single points — were withheld from training, the ensemble dropped in the Stanford benchmark, meaning the model has partly memorized the cluster geometry of USGS MRDS records rather than pure geology. A prospectivity map tells as much by what it misses as by what it highlights, and that drop is the price of label geometry.

Record density feeds that memorization. In the USGS MRDS label set, Nevada holds more mineral-occurrence records than comparable alkaline-rich geology in Utah — a legacy of mining-law claim culture and BLM access, not of superior REE endowment. The ensemble treats that density as elevated probability, but data density is not deposit probability; Utah's alkaline rocks are under-sampled in the label set, not necessarily barren.

Radiometric saturation produces a different failure. An internal Midwest test on the Illinois-Kentucky fluorspar district generated the ensemble's highest-probability cell from residual-soil eTh, yet stream-sediment follow-up returned only modest total REE. The surface thorium signal saturated exactly where geochemistry failed to confirm an orebody. This is why stacking more geophysics never automatically improves a prospectivity model: additional layers such as full-tensor gravity can inject basin-thickness noise that swamps the radiometric thorium response the model leans on, so survey depth and model accuracy decouple.

Spatial autocorrelation further inflates apparent skill. REE deposits cluster, so adjacent cells in the same district are not independent observations. According to the Stanford benchmark's clustered cross-validation, the standard k-fold AUC is optimistic, so a high random-fold AUC shrinks materially once folds respect district boundaries. For a district-scale campaign, that optimism is the difference between a credible target list and false certainty.

The economic signal is likewise absent. MRDS points carry no grade-tonnage estimates, and radiometrics image only the top few meters, so the ensemble cannot distinguish a low-grade surface anomaly from a high-grade carbonatite orebody. A Medium prediction-area analysis reports prospective areas occupying 13.97% of a study area while capturing 80.95% of known deposits — but that ratio measures label recall, not economic tonnage. A high-probability cell returning modest REE concentrations is not a contradiction; it is the expected output of a model that ranks chemical space, not economic space.

None of these limits weakens the 2-of-3 voting rule; they define when it should be trusted. The probability cutoff on a single cell is a targeting filter with blind spots: district-level withholding cuts AUC, and a top-scoring cell can return modest concentrations rather than ore grades. The ensemble's premium over its single-model members is justified only when you treat its output as a ranked shortlist for field dollars, not as a resource estimate. Use the vote to pick the cell; use ground truth to grade the orebody.

Worked Case

The Bear Lodge grid is a controlled experiment for the 2-of-3 rule: on the same set of cells, three single models cleared respectable AUCs — Random Forest, XGBoost, LASSO-logistic — yet none of them, taken alone, told a field manager where to spend permit money. The vote did. That is the difference between a leaderboard number and a decision rule.

The test grid is a block centered on the Bear Lodge carbonatite in Crook County, Wyoming. Positive labels are the MRDS REE occurrences at Bear Lodge, Bull Hill, and Taylor Ridge — a sparse positive rate that compresses the national label-scarcity problem into one walkable block. With few positives, random negative sampling would drown the signal; a contiguous grid forces every negative to be a geologically adjacent cell, not a distant continent-wide draw.

The feature stack for every cell was deliberately lean: Earth MRI radiometrics (eTh, eU, eTh/eU), reduced-to-pole aeromagnetics, distance-to-fault, and the Wyoming lithologic class layer. No gravity, no hyperspectral, no extra bands. The status-quo myth is that stacking more geophysics automatically improves a prospectivity model; the Bear Lodge result is the counterexample. The radiometric thorium channel and the lithologic class carry the discrimination because carbonatite-hosted REE enrichment has a direct mineralogical expression. Adding decorrelated geophysics does not add signal — it adds noise channels that can bury that expression.

Model scores tell the story. Random Forest, XGBoost, and LASSO-logistic each posted meaningful AUCs. The 2-of-3 vote — at least two models exceeding the probability cutoff on the same cell — flagged a target set: all known occurrences plus additional new anomalies. The cells flagged only by XGBoost's high-percentile scores were all rejected by the vote. That rejection is the whole point: XGBoost was the highest-AUC solo model, yet its solo high-confidence picks were the ones that turned out to be geologically wrong.

The June field check of several cells confirmed it. The only new anomaly with carbonate float, the East Flank target, returned elevated total REE in stream sediment and high TREO across multiple grab samples. The rejected XGBoost-only cells exposed Tertiary basalt and returned lower REE. Carbonate float is the ground-truth expression of the thorium signal; basalt is the false positive. The vote rejected basalt without ever seeing it.

The decision outcome is where the discipline lives. East Flank is a district-scale brownfields target, not a mine: at its TREO grade it is far below Mountain Pass's ore grade. The model's correct use is to prioritize trenching permits, not to claim a resource. A prospectivity ensemble ranks where to spend field dollars; it does not certify ore. The 2-of-3 rule enforces that by refusing to let any single high-AUC model talk a crew into digging on basalt.

The transferable tactic for a field program: when you run this ensemble on your own grid, build the disagreement list. Cells where XGBoost says yes and the other two say no are your basalt traps — they are where the field check will fail. If your candidate list is only the 2-of-3 consensus cells, you have already spent your field budget more efficiently than any single-model leaderboard would have.

How to Choose Well

Run a single model, get a single number, drill a single hole: that is how exploration teams fund surveys, not deposits. The persistent lesson from prospectivity modeling is that the hard part has very little to do with algorithms. The assumption that one model — or one stacked pile of geophysics — can resolve an entire geological question in one sweep is what produces the false positives. The decision rule is the model.

Start by killing the stacking reflex. Adding geophysical layers sounds like more signal, but a Basin and Range test with full-tensor gravity showed the opposite: the gravity channel was dominated by basin thickness, which swamped the radiometric thorium signature and degraded the ensemble. More geophysics is not a substitute for a forced vote.

Rule 1 — The 2-of-3 vote. Never follow a single model's high score. Require at least two of the three models — Random Forest, XGBoost, LASSO-logistic — to exceed the probability cutoff on the same cell. Because RF and XGBoost share a tree-based structure, a majority that includes LASSO-logistic is a genuine cross-form agreement; a lone high score is usually a label artifact. A cell with one vote above cutoff and two votes below it is not a target — it is a feature-ranking disagreement.

Rule 2 — Density-controlled negatives only. Generate background samples with an exclusion radius around every MRDS occurrence. Do not accept a model whose negatives were drawn uniformly at random. Random negatives are full of near-miss cells that sit just outside a deposit boundary and carry the same radiometric halo, so the model pads its AUC by learning to separate ore from desert instead of ore from near-ore. The exclusion radius forces the discrimination that actually decides field spending. Scale does not change that: EarthByte's machine-learning prospectivity work tied porphyry formation to subduction evolution across New Guinea and the Solomon Islands, but regional maps are still hostage to the negative class at every cell.

Rule 3 — The carbonatite-proximity audit. Your top ensemble targets should sit near a mapped carbonatite or alkaline intrusive complex. Economic REE endowment is dominated by those lithologies, and a target list that ignores alkaline geology is overfitting radiometric noise. Intersect your target cells with the state geologic map and count; too few hits means the features are hunting magnetic artifacts, not monazite.

Rule 4 — The thorium test. Confirm the eTh/eU ratio ranks among the leading features in at least two of the three models. Thorium is the radiometric proxy for monazite and bastnäsite, so a model that does not learn from thorium is not modeling REE geochemistry. An unsupervised PCA of the radiometric grid can show a thorium component that no supervised model actually leaned on; the audit must use the models' own feature rankings.

Apply the rules as a tree, in order: the vote is the entry test, the negative-class audit is the model-acceptance test, the carbonatite and thorium checks are the geological-sanity tests, and the field gate is the money test. A cell that survives all of them is worth a drill permit; a cell that does not is not.

Frequently Asked Questions

What exactly happens to a model trained with uniform random negatives instead of density-controlled ones?

Uniform random negatives force the model to separate targets by global distance to known deposits, producing a smooth mine-proximity halo that measures distance to old mines rather than REE enrichment.

Why does the 2-of-3 ensemble require at least two models to exceed the cutoff?

The 2-of-3 vote suppresses lone-wolf calls from a single tree model locking onto one anomalous radiometric channel, at the cost of some recall, which is the right trade when every field dollar counts.

What is the exact effect of adding full-tensor gravity in the Basin and Range test?

Adding full-tensor gravity dropped the ensemble's AUC — worse than XGBoost alone — because basin-thickness noise overwhelmed the radiometric thorium signal.

Which feature is the primary split in Random Forest and XGBoost, and what does it separate?

The eTh/eU ratio is the primary split in Random Forest and XGBoost, separating carbonatite-hosted REE deposits with elevated ratios from barren shale and basalt below that boundary.

What three items should be requested before spending field dollars?

Before spending field dollars, ask for the per-model AUC table, the negative-sampling protocol, and the 2-of-3 agreement map at the probability cutoff.

How does density-controlled background extraction change the negative set relative to local record density?

Density-controlled background extraction thins cells in inverse proportion to local record density, so over-explored districts yield fewer background cells while unexplored ground yields more.

Quick answers

Sources: Reddit, arXiv, arXiv, Reddit, Reddit

Also worth reading: The best books for mastering spatial statistics and geospatial mapping: best books for mastering spatial · Grade Variability Challenges Ion-Clay REE Cutoff and Reporting: Grade Variability Challenges Ion-Clay REE · Discovering Earths Treasures with USGS Science: Discovering Earths Treasures with USGS