View source for RLMF

{{Short description|Proposed reinforcement learning with rewards measured by molecular sensors}}
{{Hatnote|Not to be confused with reinforcement learning with metacognitive feedback, a 2026 method for calibrating language models that is also abbreviated RLMF.}}
{{Infobox
| title = RLMF
| image = Molecular Self-Play animation.webp
| caption = "Molecular Self-Play" (July 2026), the company's animation of a pore that traps a molecule, reads it repeatedly and releases it, steered by the measured current
| label1 = Full name
| data1 = Reinforcement Learning with Molecular Feedback
| label2 = Variant
| data2 = Reinforcement Learning from Molecular Feedback
| label3 = Proposed by
| data3 = [[Molecular Reality Corporation]]
| label4 = Public since
| data4 = June 29, 2026
| label5 = Reward source
| data5 = The instrument
| label6 = Status
| data6 = "a hypothesis"
}}
'''RLMF''', '''Reinforcement Learning with Molecular Feedback''', is a proposal by [[Molecular Reality Corporation]] for training artificial intelligence systems with rewards that come from measurements made by molecular sensors. The name adapts [[reinforcement learning from human feedback]] (RLHF), in which a model learns from people's ratings of its output. The company's web copy introduces it in one sentence: "Reinforcement Learning with Molecular Feedback (RLMF) is the obvious next step as AIs require more and more real biological data to solve diseases."<ref name="rlmf">{{cite web |title=RLMF |website=molecularreality.com |publisher=Molecular Reality Corporation |date=2026-06-29}}</ref>

The proposal has two parts. In the larger one, AI models of biology would be trained against data from the company's planned network of sensors. In the more specific one, a model would learn to control a [[solid-state nanopore]] instrument, choosing voltages and pore geometries, with rewards computed from the instrument's own signals. The company's guide is explicit about its status: "Both are proposals, not products, and I want that said first", and of RLMF, "It is a hypothesis. We are collecting the data that would test it."<ref name="faq">{{cite web |url=https://molecularreality.com/FAQ |title=Keltar's Field Guide to Molecular Reality |publisher=Molecular Reality Corporation |date=August 2026}}</ref>

== Name ==
Company materials expand the acronym two ways. The public web copy and the guide say "with Molecular Feedback"; other company documents say "from Molecular Feedback", the preposition RLHF uses.<ref name="rlmf" /><ref name="records">{{cite web |title=Company records |publisher=Molecular Reality Corporation |date=October 2026}}</ref>

A tab labeled RLMF was added to the company's website on June 29, 2026.<ref name="records" /> On June 30, 2026, researchers at Yale University and Google Research posted a preprint introducing "reinforcement learning with metacognitive feedback (RLMF)", in which a language model's rankings are refined according to how well it judges its own performance.<ref name="liu">{{cite web |url=https://arxiv.org/abs/2606.32032 |title=Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs |last1=Liu |first1=Gabrielle Kaili-May |website=arXiv |date=2026-06-30}}</ref> The two proposals share nothing but the letters.

== Background ==
RLHF trains a reward model on human judgments, then trains a policy to score well against it. The method was demonstrated on simulated robots and video games in 2017, using people's choices between pairs of short clips,<ref name="christiano">{{cite web |url=https://arxiv.org/abs/1706.03741 |title=Deep reinforcement learning from human preferences |last1=Christiano |first1=Paul |website=arXiv |date=2017-06-12}}</ref> and was applied to instruction-following language models in 2022.<ref name="ouyang">{{cite web |url=https://arxiv.org/abs/2203.02155 |title=Training language models to follow instructions with human feedback |last1=Ouyang |first1=Long |website=arXiv |date=2022-03-04}}</ref> Several later methods keep the reinforcement learning and replace the human. A 2025 preprint from a protein-engineering group called its version RLXF, "Reinforcement Learning from eXperimental Feedback", and used laboratory measurements of protein function as the reward.<ref name="blalock">{{cite web |url=https://doi.org/10.1101/2025.05.02.651993 |title=Functional alignment of protein language models via reinforcement learning |last1=Blalock |first1=Nathaniel |website=bioRxiv |date=2025-05-08}}</ref> [[Periodic Labs]], founded in 2025 to search for new materials with automated laboratories, describes its approach the same way; its co-founder Liam Fedus said in October 2025 that "our reinforcement learning environments literally derive from the environment, from our physical labs."<ref name="latent">{{cite web |url=https://latent.space/p/periodic |title=Synthesis Superintelligence: from Semiconductors to Superconductors |website=Latent Space |date=2025-10-08}}</ref>

Molecular Reality's version applies the substitution to molecules. Its web copy argues that "Even the most intelligent AI systems will need to somehow measure the world of molecules" and that "it will take a whole new level of molecular sensing infrastructure to create the data needed."<ref name="rlmf" /> The same argument underlies the company's tagline, "[[Everybody's racing to build the brain|everybody's racing to build the brain]], we're building the eye", in which "the brain" means AI laboratories working on biology and materials, Periodic Labs among them, and "the eye" means sensors.<ref name="p3">{{cite news |title=The Startup That Says Everyone Else Is Building the Brain |work=The Picoampere |author=Dev Okonkwo-Hale |date=2026-08-21 |url=/press/picoampere/2026/08/21/brain-eye}}</ref>

== Origins in the company ==
An earlier form of the idea appears in the company's December 2022 technical overview, which planned to "adjust the game mechanics to encourage enough data gathering to train advanced deep reinforcement learning models that will learn to optimize translocation control using combinations of ping-pong, sliding occlusion, different pores, different salt and buffer conditions, and several other parameters."<ref name="overview">{{cite web |title=2022 sci-tech overview |author=Kemmishtree |website=EQBio-OS |date=2026-03-28}}</ref> In that plan the data would come from volunteers playing games on home instruments, an approach the company calls [[experigaming]].

By 2026 the inspiration named in company study materials was self-play, in which a game-playing program generates its own training data by playing against itself, as AlphaZero did for chess and Go. A white paper of July 2026 titles its section on control "the pore as a game the model learns to play", on the reasoning that an instrument whose controls are moves can produce its own labeled examples.<ref name="records" /> On July 24, 2026 the company published an animation called [[Molecular Self-Play]], captioned "TRAP · SENSE · RELEASE · REPEAT": "A programmable solid-state pore traps a single construct, ping-pongs the analyte through the sensing zone to tomographically sub-sample different conformations, then releases it".<ref name="msp">{{cite web |url=https://molecularreality.com/MSP |title=Molecular Self-Play |publisher=Molecular Reality Corporation |date=2026-07-24}}</ref>

== Proposed design ==
=== Rewards ===
A reward based on the identity of the molecule would be circular, since the identity is what the instrument is trying to learn. Company documents therefore propose rewards "measured by the instrument itself, never human annotation": the rate at which molecules are captured, the agreement between repeated reads of one molecule, the signal-to-noise ratio, and the information a measurement adds.<ref name="records" /> The last is the quantity that statistical theory recommends for choosing experiments; it goes back to a 1956 paper by Dennis Lindley on measuring the information an experiment provides.<ref name="lindley">{{cite journal |last1=Lindley |first1=D. V. |title=On a measure of the information provided by an experiment |journal=The Annals of Mathematical Statistics |volume=27 |issue=4 |pages=986-1005 |year=1956 |doi=10.1214/aoms/1177728069}}</ref> Rewards that do not depend on identity are also meant to resist reward hacking, in which a model scores well without doing what was intended.<ref name="records" />

Ground truth would be added in stages, which the documents describe as a ladder: weak labels from operators' notes, then samples of known composition, then engineered DNA constructs carrying barcodes, a process summarized as "using labels to get free of labels".<ref name="records" />

=== Control ===
The proposed controller works at three speeds. Fixed logic in the instrument's firmware reacts within microseconds, a small model on the device adjusts thresholds within milliseconds, and a larger model sets the strategy over seconds or minutes. "The slow model never actuates the piezo at microsecond speed; it sets the ''policy''."<ref name="records" /> The actions available are the instrument's settings: the voltage, its timing, the salt and, in the company's [[sliding occlusion]] designs, the shape of the pore.

The company's public essay [[The Nanopore World Model]] limits the claim. Holding and rereading a single molecule works only when the molecule is confined, by tethering, by two pores pulling against each other, or in a tiny bridge between pores; a free small molecule diffuses away in microseconds. Elsewhere the controller would choose conditions for the next population of molecules, a regime the essay summarizes as "the distribution confesses, not the molecule."<ref name="wm">{{cite web |url=https://molecularreality.com/nanopore-world-model |title=Keltar's Critique v2: The Nanopore World Model (Hardened) |publisher=Molecular Reality Corporation |date=2026-08}}</ref>

== Related work ==
Nanopore instruments already make decisions in real time. In 2009 a group at the University of California, Santa Cruz used programmable hardware to change the voltage within a fraction of a millisecond of detecting a DNA-enzyme complex.<ref name="wilson">{{cite journal |last1=Wilson |first1=Noah A. |title=Electronic control of DNA polymerase binding and unbinding to single DNA molecules |journal=ACS Nano |volume=3 |issue=4 |pages=995-1003 |year=2009 |doi=10.1021/nn9000897}}</ref> Commercial sequencers can reject unwanted DNA by reversing the voltage once the first stretch of a molecule has been read, a method published in 2016,<ref name="loose">{{cite journal |last1=Loose |first1=Matthew |title=Real-time selective sequencing using nanopore technology |journal=Nature Methods |volume=13 |pages=751-754 |year=2016 |doi=10.1038/nmeth.3930}}</ref> and a 2023 method chooses which molecules to keep by Bayesian experimental design.<ref name="weilguny">{{cite journal |last1=Weilguny |first1=Lukas |title=Dynamic, adaptive sampling during nanopore sequencing using Bayesian experimental design |journal=Nature Biotechnology |volume=41 |pages=1018-1025 |year=2023 |doi=10.1038/s41587-022-01580-z}}</ref> In 2025 a study of solid-state pores combined recordings at several fixed voltages with machine learning to tell apart proteins in mixtures; the voltages were set in advance.<ref name="akita">{{cite journal |last1=Akita |title=Voltage-matrix nanopore profiling for the discrimination of protein mixtures |journal=Chemical Science |volume=16 |pages=18607-18615 |year=2025 |doi=10.1039/d5sc05182g}}</ref> A literature check by the company in October 2026, covering work from 2015 on, found no published case of a learned policy controlling the voltage or geometry of a nanopore in real time.<ref name="records" />

== Status ==
No RLMF experiment has been reported. The company's machine-learning results so far come from supervised classification: a Conformer network that reached 85.4 percent accuracy on held-out experiments from a public DNA-barcode dataset, against 81.0 percent for a retrained convolutional network, a result described as unpublished.<ref name="records" /> The guide describes the company's present contribution as data collection.<ref name="faq" />

== See also ==
* [[Machine learning at Molecular Reality]]
* [[Active sensing]]
* [[Self-driving laboratory]]
* [[Ping-pong sensing]]

== References ==
{{Reflist}}

{{Brain labs}}
{{Molecular Reality}}

[[Category:Reinforcement learning]]
[[Category:Molecular Reality Corporation]]
[[Category:Proposals]]