Artificial intelligence is currently dominated by systems built through large-scale pretraining. Frontier models absorb enormous collections of text, images, software code, audio, and video before they are deployed. Once trained, they can generate remarkably sophisticated responses, but their central learned parameters ordinarily remain largely fixed during everyday use. They can process new information placed in a prompt, retrieve current material from databases, and use external tools, yet they do not normally reconstruct their internal knowledge continuously from every experience.

OaK Lab is pursuing a fundamentally different model of machine intelligence.

Founded by reinforcement-learning researcher Richard S. Sutton and his longtime collaborator Khurram Javed, OaK Lab is developing what it calls the OaK architecture, short for Options and Knowledge. Its objective is to build computational agents that learn continuously from their own streams of experience, construct increasingly useful abstractions, develop reusable behavioral skills, predict the consequences of those skills, and plan in environments too complex to be fully represented in advance.

OaK Lab describes its goal as discovering and implementing algorithms that allow agents to achieve goals in “big worlds.” Its proposed agents would learn in real time, operate with a batch size of one, avoid dependence on stored replay data, and remain capable of revising their internal structures throughout their operational lives. The laboratory’s long-term engineering aspiration is extraordinarily ambitious: a trillion-parameter agent that learns and plans in real time while consuming approximately 20 watts of power. (⁠oaklab.ai)

This is not merely a proposal for a more efficient neural network. It is an attempt to rethink how an intelligent machine should be organized.

Richard Sutton and the foundations of reinforcement learning

Richard S. Sutton is one of the central figures in the development of modern reinforcement learning. His work began in the late 1970s at the University of Massachusetts Amherst, where he studied under Andrew G. Barto. Together, Sutton and Barto helped transform ideas drawn from psychology, animal learning, control theory, and dynamic programming into a general computational framework in which an agent learns through interaction with an environment.

In reinforcement learning, an agent observes aspects of its environment, chooses actions, and receives numerical reward signals indicating how well its behavior is serving its objective. The agent’s task is not merely to obtain the largest immediate reward but to learn behavior that maximizes reward accumulated over time.

Sutton’s most influential contributions include temporal-difference learning, methods for learning value functions, policy-gradient methods, neural-network representations for reinforcement learning, and architectures that integrate learning with planning. His work also helped establish the idea that an intelligent system should learn predictions directly from an unfolding stream of experience rather than depend entirely on a complete model supplied by its designers.

In 2025, the Association for Computing Machinery announced Sutton and Barto as the recipients of the 2024 ACM A.M. Turing Award, commonly regarded as the highest distinction in computer science. The award recognized them for developing the conceptual and algorithmic foundations of reinforcement learning. ACM specifically credited their work on temporal-difference learning, policy-gradient methods, neural representations, and integrated learning-and-planning architectures. (⁠awards.acm.org)

Sutton has held research and academic positions at AT&T Bell Laboratories, the University of Alberta, DeepMind, the Alberta Machine Intelligence Institute, and Keen Technologies. His recent research has increasingly emphasized the difference between intelligence acquired from an agent’s own life and capabilities extracted from human-produced datasets.

That distinction is central to OaK Lab.

Sutton does not deny that large language models can be useful or impressive. His position is more fundamental: learning patterns from records of human experience is not equivalent to having experience. A model trained on internet data can reproduce, combine, and generalize from information created by other agents, but it does not necessarily possess the continuing sensorimotor relationship with the world that Sutton considers essential to general intelligence.

The question Oak Lab asks is therefore not simply, “How can a model know more?” It is, “How can an agent continue becoming more capable through its own existence?”

Khurram Javed and learning how to learn online

Khurram Javed represents a younger generation of researchers working within the reinforcement-learning tradition established by Sutton and his colleagues. Javed studied computing science at the University of Alberta and worked within the university’s reinforcement-learning research community. His graduate research concentrated on continual learning, meta-learning, online adaptation, recurrent computation, representation learning, and methods for allowing neural systems to change without rapidly destroying previously acquired capabilities.

Javed’s work addresses one of the central problems confronting any lifelong learning system: the stability-plasticity dilemma.

A learner must remain plastic enough to acquire new knowledge when its environment changes. At the same time, it must remain stable enough to preserve important earlier knowledge. A system that changes too slowly becomes obsolete. A system that changes too quickly may overwrite useful capabilities whenever it encounters new data.

In research with Martha White, Javed explored online-aware meta-learning, in which a representation is trained not merely to perform well on a current task but to remain suitable for future online updates. The purpose is to construct internal representations whose parameters can continue changing with less destructive interference.

Javed has also coauthored work with Sutton and other researchers on scalable real-time recurrent learning. Such research investigates how a continually operating agent might learn temporal relationships without relying on the computationally expensive training procedures typically used for large recurrent neural networks.

At OaK Lab, Javed’s work complements Sutton’s long-standing theoretical program. Sutton supplies much of the conceptual architecture for reinforcement learning, temporal abstraction, predictive knowledge, and agent-centered intelligence. Javed contributes recent work on continual adaptation, meta-learning, computational efficiency, and neural systems designed for real-time operation.

Together, they are pursuing a form of artificial intelligence in which learning is not a preparatory phase that ends before deployment. Learning is the continuing activity of the machine.

The alternative to frontier-model development

The dominant frontier-model strategy can be summarized as follows:

1. Collect or generate an enormous dataset.
2. Train a large neural network through repeated passes over batches of that data.
3. Freeze or substantially stabilize the resulting parameters.
4. Deploy the model for inference.
5. Improve it later through additional training, fine-tuning, retrieval systems, or replacement with a newer version.

This approach has produced extraordinary advances. It has also created several structural limitations.

First, the training data largely represents other people’s experiences. A language model learns from text written by humans, code produced by programmers, images captured by cameras, and judgments recorded by annotators. Even when synthetic data is added, it is often generated by models whose capabilities ultimately trace back to human-curated material.

Second, the model’s training and deployment are separated. The system may process new information after deployment, but most of that information does not permanently alter its central parameters in real time.

Third, large-scale batch training requires substantial memory, computation, communication bandwidth, and energy. Training often depends on repeatedly presenting stored examples to the model.

Fourth, a pretrained model may perform well within the statistical boundaries represented by its data yet struggle when confronted with an evolving environment requiring persistent adaptation.

OaK Lab’s approach does not attempt merely to scale this process more efficiently. It begins from a different premise: an intelligent machine should be an agent living within a continuing stream of interaction.

The agent should perceive, act, receive consequences, update predictions, change behavior, create new internal features, learn new skills, revise models, discard obsolete abstractions, and plan continuously. There should be no absolute division between training and inference because the system should never stop learning.

The Big World Hypothesis

OaK Lab’s research is guided by what Sutton and Javed call the Big World Hypothesis.

A big world is an environment whose complexity vastly exceeds the computational, sensory, and representational capacity of the agent operating within it. The agent cannot observe everything, remember everything, calculate every possible consequence, or construct a complete model of every relevant process.

The physical world is big in this sense. So are societies, economies, biological systems, industrial facilities, transportation networks, and environments containing other intelligent agents.

A robot operating in a factory, for example, cannot represent every microscopic feature of the building, every possible movement of every worker, every potential machine failure, every vibration, every future production schedule, and every physical interaction that could occur. It must select what to notice, what to predict, what to retain, and what to ignore.

The Big World Hypothesis therefore treats intelligence as a problem of selective, resource-bounded adaptation.

The challenge is not to create a perfectly complete internal copy of the world. Such a copy would generally be impossible. The challenge is to construct partial knowledge that is useful for action and to continually replace that knowledge as the environment and the agent’s objectives change.

This view directly challenges the idea that intelligence can be achieved simply by making a sufficiently large static model. Even if computing capacity grows, the available world may grow with it. Better sensors can produce more data. Faster processors can permit more complex interactions. Other agents can also become more sophisticated. The relevant environment can continue to exceed the capacity of any individual learner.

An intelligent agent must therefore manage its own limited computational resources. It must learn not only how to act but also what to represent, what to predict, what to practice, and what to forget.

The OaK architecture: Options and Knowledge

The name OaK refers to two major categories of learned abstraction.

Options are temporally extended behaviors. Instead of selecting only one primitive action at a time, an agent can learn a reusable sequence of actions that accomplishes a recognizable result.

A primitive action for a robot might be increasing electrical current to a particular motor for a fraction of a second. An option might be moving toward a door, grasping an object, loading a component, stabilizing a tool, or navigating to a charging station.

An option generally includes:

* conditions under which it can begin;
* an internal policy specifying what actions to take;
* and a termination rule specifying when the behavior is complete.

Knowledge consists largely of learned predictions about the agent’s sensory signals, rewards, states, options, and future experience. Rather than requiring all knowledge to be expressed in human language, OaK treats knowledge as predictive structure grounded in the agent’s interaction with the world.

An agent might learn that a particular visual pattern predicts contact, that a vibration predicts equipment failure, that executing a navigation option will probably lead to a doorway, or that using a particular skill under certain conditions will produce a delayed reward.

The OaK architecture seeks to connect these two categories. The agent discovers features, turns selected features into subtasks, learns options for accomplishing those subtasks, constructs models of the options’ consequences, and uses the models for planning.

Just as importantly, it must evaluate whether each learned element remains useful. Features, options, and models cannot accumulate indefinitely. Components that no longer contribute to prediction, control, or planning must eventually be weakened, replaced, or removed.

The Alberta Plan: a 12-stage research agenda

OaK did not emerge as an isolated concept. It is situated within a broader program described by Sutton, Michael Bowling, and Patrick Pilarski in The Alberta Plan for AI Research.

The Alberta Plan is a long-term agenda for understanding and creating computational intelligence. Its authors describe it as a five-to-ten-year basic-research program rather than a plan for immediate commercial products. The objective is to develop long-lived agents that interact with worlds much more complex than themselves, learn continually, construct models, and plan. The authors also emphasize that the roadmap is provisional: later stages may change as earlier research produces new discoveries. (⁠arXiv)

The plan consists of 12 stages that build progressively from basic online learning toward integrated continually learning intelligence.

Stage 1: Continual supervised learning with given features

The first stage begins with the simplest controlled setting. The learner receives an ongoing sequence of examples represented by features selected in advance. It must make predictions and update itself incrementally after each new observation.

This differs from ordinary batch learning. The agent does not collect a complete training dataset and repeatedly process it in randomized groups. Each example arrives once as part of a continuing stream.

The principal research problem is to develop methods that learn accurately and stably under nonstationary conditions. Relationships may change over time, and the learner must track those changes without starting over.

This stage provides a foundation for later continual-learning algorithms because it isolates the basic problem of adjusting parameters from one experience to the next.

Stage 2: Continual supervised learning with feature construction

The second stage removes the assumption that the correct features are supplied by a human designer.

The system must construct or discover internal features that make prediction easier. In a neural network, this means learning intermediate representations rather than adjusting only a final prediction layer.

This immediately introduces the stability-plasticity problem. Internal features must evolve as the environment changes, but changing them can disrupt every prediction that depends on them.

A successful feature-construction mechanism must determine:

* which patterns are worth representing;
* which features remain useful;
* when new features should be created;
* when old features should be modified;
* and when obsolete features should be removed.

This is a major transition. The agent is no longer merely fitting an answer. It is beginning to determine the internal language in which its knowledge will be expressed.

Stage 3: Continual prediction through generalized value functions

The third stage moves from ordinary supervised targets to predictions about future experience.

Sutton and his collaborators use generalized value functions, or GVFs, to represent questions about temporally extended outcomes. A GVF might predict:

* how much cumulative vibration will occur;
* whether a sensor threshold will be crossed;
* how long a battery will remain usable;
* what observation will follow a particular behavior;
* or how much reward will accumulate under a specified policy.

This approach treats predictive knowledge as a large collection of learned questions and answers grounded in experience.

The agent does not need a complete symbolic world model. It can instead learn many partial predictions that become useful for control and planning.

Stage 4: Continual actor-critic control

Prediction alone is not intelligence. The agent must also act.

The fourth stage introduces a continuing actor-critic architecture. The actor learns which actions to select. The critic estimates how successful the resulting behavior is likely to be.

Both components learn online. The critic uses reward and subsequent experience to evaluate the actor’s decisions. The actor then changes its policy in directions expected to produce greater long-term reward.

The challenge is to achieve stable control while the agent’s predictions, features, policy, and environment may all be changing simultaneously.

Stage 5: Average-reward continual prediction

Many reinforcement-learning formulations divide experience into episodes with clear beginnings and endings. Real life often does not.

A factory robot, power-management system, network controller, or autonomous vehicle may operate continuously. There is no natural final state after which all calculations reset.

The fifth stage therefore emphasizes average-reward formulations, in which the agent seeks to improve the long-run rate of reward during continuing operation.

Average-reward learning introduces difficult theoretical and practical questions. The agent must distinguish between temporary gains and sustained performance. It must also learn appropriate baselines for evaluating whether current experience is better or worse than its long-term average.

Stage 6: Continuing control

The sixth stage combines average-reward prediction with action selection in continuing environments.

The objective is an agent that can control an ongoing process without requiring artificial episodic boundaries. It must learn while operating, adapt to gradual or abrupt changes, and improve long-term performance.

This is particularly relevant to real industrial systems. Water-treatment plants, electrical grids, communication networks, manufacturing lines, and robotic facilities do not conveniently reset after each experiment.

A continuing controller must learn without disrupting the process it controls.

Stage 7: Planning with learned average-reward models

The seventh stage introduces planning based on learned models.

A purely reactive agent changes behavior only after experiencing consequences. A planning agent can use an internal model to consider possible outcomes before acting.

The model need not reproduce every physical detail. It may predict only the consequences most useful for decision-making.

In the Alberta Plan, planning is closely integrated with learning. The agent learns both from real interaction and from simulated transitions generated by its model. This follows Sutton’s earlier Dyna architecture, which unifies direct reinforcement learning with model-based planning.

In a continuing environment, the model and planner must operate under average-reward objectives and limited computational resources.

Stage 8: Prototype AI I—one-step model-based reinforcement learning

The eighth stage integrates the preceding components into a first prototype agent.

This agent continually:

* observes;
* predicts;
* selects actions;
* receives reward;
* updates its features;
* learns a model;
* and uses that model for planning.

The model initially operates primarily at the level of one-step transitions. It predicts what is likely to happen immediately after a primitive action.

This prototype is significant because the research program shifts from improving isolated algorithms to testing how multiple continually changing components interact within one complete architecture.

Many difficulties may appear only after integration. A feature representation useful for prediction may be poor for control. A learned model may destabilize planning. Planning may concentrate computational effort on irrelevant states. The prototype provides a platform for identifying these interactions.

Stage 9: Search control and exploration

A model-based agent cannot plan equally about every possible state or action. It must decide where to direct its limited planning computation.

This is the problem of search control.

The agent may prioritize states associated with large prediction errors, recent changes, high uncertainty, important rewards, or likely future experience. It must also decide how to explore the real environment to obtain information its model lacks.

Search control transforms planning from indiscriminate simulation into selective computation. It determines which hypothetical experiences deserve attention.

This stage is essential under the Big World Hypothesis. Because neither the world nor the planning space can be exhaustively searched, intelligence requires disciplined allocation of computational effort.

Stage 10: Prototype AI II—the STOMP progression

The tenth stage introduces temporal abstraction through the STOMP progression:

SubTask → Option → Model → Planning

Feature construction is often placed before this sequence, producing the extended expression FC-STOMP:

Feature Construction → SubTask → Option → Model → Planning

The progression begins when the agent identifies a potentially useful feature. The feature may represent a detectable condition, predictive pattern, sensorimotor relationship, or recurring state.

The agent then creates a subtask associated with that feature. It might attempt to make the feature occur, maintain it, avoid it, or control its value.

Next, the agent learns an option—a temporally extended skill—for accomplishing the subtask.

It then learns a model predicting the option’s consequences. Instead of predicting only what happens after one primitive action, the model predicts the state, reward, duration, and relevant features expected when the entire option terminates.

Finally, the planning system incorporates the option model. It can reason using higher-level skills rather than considering every low-level action individually.

A mobile robot might therefore plan with concepts such as “move to workstation,” “retrieve container,” and “return to charging dock” instead of simulating thousands of individual motor commands.

The potential advantage is substantial. Useful temporal abstractions can extend the effective planning horizon while reducing the number of decisions that must be explicitly evaluated.

Stage 11: Prototype AI III—the OaK architecture

The eleventh stage expands STOMP into the full Options and Knowledge architecture.

At this level, the agent continually constructs and maintains a changing collection of:

* features;
* predictive questions;
* subtasks;
* options;
* option models;
* value functions;
* planning structures;
* and policies.

The architecture must determine the utility of each component. A learned feature that improves important predictions should be preserved. An option that repeatedly contributes to effective plans should receive continued computational support. A model that is inaccurate or never used should eventually lose priority.

This introduces a form of internal competition for resources.

The agent has limited memory, computation, and learning capacity. It cannot preserve every pattern it observes or every skill it attempts to learn. It must allocate resources to the structures that produce the greatest long-term value.

OaK therefore represents more than hierarchical reinforcement learning. It is a proposal for continual self-organization.

The agent constructs abstractions, tests them through experience, uses them for action and planning, and replaces them when better abstractions become available.

Stage 12: Prototype intelligence amplification

The final stage of the Alberta Plan is called intelligence amplification.

Once a continually learning agent can construct knowledge and options, the research question becomes whether the system can improve the processes by which it learns, represents, plans, and allocates computation.

This is not necessarily unrestricted recursive self-modification. It is a research program in which the agent learns to improve components of its own intelligence.

Examples might include learning:

* better step sizes;
* better feature-construction rules;
* better search-control policies;
* better methods for selecting subtasks;
* better ways of allocating memory;
* or better strategies for deciding what to learn next.

Meta-learning is central to this stage. The system does not learn only about its external environment. It also learns how its internal learning mechanisms should change.

The long-term possibility is an agent whose capacity to acquire useful knowledge improves as a consequence of its own experience.

Meta-learned step sizes

One of the most concrete mechanisms connecting Oak Lab’s current research to intelligence amplification is the use of meta-learned step sizes.

In gradient-based learning, a step size determines how much a parameter changes after an error. If the step size is too small, learning is slow. If it is too large, useful knowledge may be overwritten or the system may become unstable.

Conventional deep-learning systems often use one global learning rate or an adaptive optimizer that scales updates according to recent gradient statistics.

OaK Lab’s research direction gives individual parameters their own adaptive step sizes. These step sizes are themselves learned according to how well each parameter contributes to future performance.

A parameter representing a stable relationship may gradually receive a smaller step size, protecting it from unnecessary change. A parameter associated with a rapidly changing relationship may receive a larger step size, allowing faster adaptation.

This does not eliminate catastrophic forgetting, but it offers a principled mechanism for balancing stability and plasticity at a fine-grained level.

Learning without replay

Another defining Oak Lab objective is real-time learning without storing and replaying large datasets.

Replay buffers are common in deep reinforcement learning. They store earlier transitions and repeatedly sample them during training. Replay can improve stability and data efficiency, but it imposes memory costs and departs from strictly online learning.

OaK Lab proposes batch-size-one algorithms that process each experience as it arrives. The system should extract as much useful information as possible immediately, update its internal structures, and continue operating.

This design could provide several advantages:

* lower storage requirements;
* immediate adaptation;
* reduced dependence on centralized training;
* better privacy when raw experiences need not be retained;
* and greater compatibility with embedded or autonomous systems.

The difficulty is that replay also protects against forgetting and allows rare experiences to be reused. A no-replay architecture must preserve important knowledge through its learned features, step sizes, predictions, models, and options rather than through repeated access to stored examples.

Whether this approach can scale to extremely large systems remains an open research question.

Event-driven computation and the 20-watt objective

OaK Lab states that batch-size-one learning combined with event-driven neural networks could reduce computation and energy consumption by multiple orders of magnitude. Its ultimate target is a trillion-parameter agent that learns and plans in real time using 20 watts. (⁠oaklab.ai)

Meeting that goal would require far more than more efficient software.

A trillion conventional parameters would require enormous memory capacity and bandwidth. Moving parameters between memory and processing units can consume more energy than the arithmetic performed on them.

A 20-watt system would therefore probably need:

* sparse activation, so only a tiny portion of the network operates at any moment;
* local learning, so updates do not require global backpropagation through the entire system;
* event-driven processing;
* specialized low-power hardware;
* highly compressed or low-precision parameters;
* in-memory or near-memory computation;
* and aggressive management of which features and options remain active.

The human brain provides inspiration because it operates at roughly this power level while supporting continual perception, action, learning, and planning. But the comparison should not be interpreted as evidence that Oak Lab has already identified how to reproduce brain-level efficiency.

The 20-watt objective is a research north star, not a demonstrated capability.

What OaK Lab could contribute to artificial intelligence

OaK Lab’s greatest importance may lie in shifting attention from pretrained artifacts to lifelong computational agents.

Large language models have demonstrated that scaling data, models, and computation can produce broad capabilities. OaK Lab asks what happens after a system enters the world.

Can it learn permanently from one encounter?

Can it adapt without being retrained from scratch?

Can it invent useful intermediate goals?

Can it discover reusable skills?

Can it plan through abstractions it constructed itself?

Can it discard obsolete concepts?

Can it continue improving while using bounded memory and energy?

These questions are especially important for robotics, autonomous systems, industrial control, scientific experimentation, adaptive infrastructure, and machines expected to operate for years in changing environments.

The most plausible future may combine rather than simply choose between foundation models and continual reinforcement-learning systems.

A pretrained language or multimodal model could provide broad initial representations, communication abilities, and accumulated human knowledge. An OaK-like architecture could add persistent learning, embodied experience, skill construction, planning, and adaptation.

In that hybrid system, pretraining would provide a beginning rather than a completed intelligence.

A scientific project rather than a finished theory

The Alberta Plan’s authors are explicit that their roadmap contains uncertainties and gaps. The early stages are more concrete than the later ones, and the architecture may change as research progresses. (⁠arXiv)

Several fundamental problems remain unresolved:

* preventing destructive interference in large continually changing networks;
* selecting useful features and subtasks;
* ensuring that self-generated objectives remain aligned with externally intended goals;
* planning accurately with incomplete models;
* exploring safely in physical environments;
* evaluating agents whose knowledge depends on their entire histories;
* protecting online learners from manipulation and poisoned experience;
* and scaling feature, option, and model management without overwhelming available computation.

OaK should therefore not be described as a completed artificial-general-intelligence system. It is a structured research hypothesis about the components such a system might require.

Its scientific value lies partly in its specificity. Instead of treating general intelligence as a mysterious consequence expected to emerge automatically from greater scale, the Alberta Plan identifies a sequence of technical problems that can be studied individually and then integrated.

The program begins with the most elementary problem—learning one prediction from an ongoing stream—and progresses toward agents that construct their own abstractions and improve aspects of their own learning.

The central idea

Oak Lab’s work rests on a deceptively simple principle:

Intelligence should be understood as an ongoing relationship between an agent and its experience.

A genuinely adaptive machine should not merely contain a compressed record of the past. It should remain capable of learning from the present.

The OaK architecture seeks to create machines that do not just answer questions from previously learned patterns but continually discover what is worth predicting, what is worth controlling, what skills are worth preserving, and what knowledge is worth using for future decisions.

Whether OaK Lab can achieve its most ambitious goals remains uncertain. No complete OaK agent has yet demonstrated human-level general intelligence, and the trillion-parameter, 20-watt system remains aspirational.
 Nevertheless, the project poses one of the most important challenges to the current direction of artificial intelligence. Instead of asking how large a model must become before it appears intelligent, Oak Lab asks how a bounded machine can remain intelligent throughout a lifetime in a world it can never completely know.

References

1. Oak Lab. (2026). Mission: Achieving goals in big worlds. Oak Lab. https://oaklab.ai/mission
2. Sutton, R. S., Bowling, M., & Pilarski, P. M. (2022). The Alberta Plan for AI research. arXiv. https://doi.org/10.48550/arXiv.2208.11173
3. Association for Computing Machinery. (2025). Andrew Barto and Richard Sutton receive the 2024 ACM A.M. Turing Award for developing the conceptual and algorithmic foundations of reinforcement learning. ACM.Uewdreftrrrf