Official implementation of Do Language Models Consistently Encode the Current Year?
A consistent concept of the current time is important for temporal reasoning, yet how language models represent the current time is not well understood. We contribute two tasks that probe the current year in conceptually distinct ways: an associative task, which infers the current year from verb tense, and a declarative task, which directly queries for the current year. Both tasks estimate current years within one year of the post-training data cutoff of instruction-tuned language models. For base models, predictions on the associative task serve as a strong proxy for the pre-training data cutoff, with an average error of only 10 months across 13 models. However, their internal mechanisms diverge: the associative task uses mechanisms similar to factual recall, while the declarative task lacks consistent causal pathways. This divergence poses a challenge for updating the current year in language models. None of prompting, SFT, or weight editing succeed in shifting the associative and declarative years simultaneously. Prompting updates the declarative year (94.6% success across 351 target years) but leaves the associative year nearly unchanged (1.7% success). Year-shifted SFT also fails to shift the associative year, matching the target year in only one of eight models. Weight editing, while effective for both tasks individually, does not generalize across both. Overall, our results show that the current year is not consistently encoded in language models: The associative notion, deeply ingrained in linguistic structures learned in pre-training, uses different causal mechanisms and resists the same modifications that easily shift the declarative notion learned in post-training.
This repository contains the code and data for the paper.
Create the environment and install dependencies:
conda create -n current-year python=3.10 -y
conda activate current-year
pip install -r requirements.txt
requirements.txt pins transformers==4.49.0, which is required: experiments.py refuses to run
the OLMo system-prompt sweeps on any other version, because OLMo's tokenizer routing changed
across releases and would move the reported DCY/ACY. Two appendix experiments need newer
transformers than that and cannot share the environment — they evaluate Gemma-3 and Qwen-3.5,
which use non-OLMo prompt formats and so are not covered by that guard:
| environment | needed for |
|---|---|
requirements.txt | everything except the two rows below |
requirements-gemma.txt | acy_tense_before_year's Gemma-3 models; update_prompting's google/gemma-3-* rows |
requirements-qwen.txt | update_prompting's Qwen/Qwen3.5-* rows |
Download the NLTK sentence tokenizer, used for the co-occurrence counts over the pre-training data:
python -c "import nltk; nltk.download('punkt_tab')"
Each experiment writes to experiments/<name>/. Run one, or list them all:
python experiments.py --list
python experiments.py -e acy_distribution
| Experiment | Description |
|---|---|
acy_distribution | Figure 1 — ACY tense distribution across 13 base LMs |
acy_across_models | Table 1 — ACY and ACYM against documented cutoffs |
acy_prompt_variants | Figure 2 — alternative carrier prompts and verb sets |
acy_tense_before_year | Figure 3 — tense marker before the year |
acy_training_data_ngram | n-gram and co-occurrence count models over the pre-training data, and their cross-entropy table |
acy_factual_recall | tense after event prompts with no explicit year |
acy_factual_recency | year implied by "most recent X" answers |
acy_knockouts | attention and MLP knockouts on the ACY prompt, measured by TSM |
acy_year_subspace_train | Figure 4 — trains the year-type subspace (5 positions × 18 layers) |
acy_year_subspace_heatmap | Figure 4 — IIA heatmap, reads the runs above |
acy_subspace_controls | Table 6 — five controls on the subspace |
cutoff_perplexity | relative perplexity over dated Wikipedia snapshots |
dcy_across_training | Table 2 — DCY, variance and confidence across training stages |
dcy_knockouts | Figure 5 — declarative knockouts, measured by YSM |
training_data_year_counts | year frequency in pre-training and SFT data |
training_dynamics | §3.3 — ACY and DCY across OLMo2 checkpoints, both model families |
update_prompting | Table 3 — naming a target year in the system prompt, across model families and scales |
update_sft_shifted | Table 3 — fine-tuning on year-shifted SFT data |
update_rome_dcy | Table 3 — ROME edit targeting the DCY |
update_memit_acy | Table 3 — MEMIT edit targeting the ACY |
The two acy_year_subspace_* experiments are staged: training writes per-position results, then
the heatmap reads them. acy_knockouts, dcy_knockouts, update_prompting and the weight-editing experiments reuse
their intermediate per-item results across runs, since each is a sweep of hundreds or thousands of
forward passes. Delete the relevant files under experiments/<name>/ to force a recomputation —
in particular update_prompting validates its cache against library versions and year coverage but
not against the prompt text, so edit a prompt and you must clear *_sys_sweep.json by hand.
The weight-editing code in src/editing.py implements ROME and MEMIT
(Meng et al., MIT). The perplexity experiment replicates the effective-cutoff method
of Cheng et al. using their dated_data pipeline.
We release the SFT models from §4.3 and §5, along with the datasets they were trained on. All are OLMo2-1B fine-tuned on modified Tulu3, the mixture OLMo2-1B itself used for SFT.
| Variant | Model | Dataset |
|---|---|---|
| SFT control | suzeva/olmo2_1b_sft_control | unmodified Tulu3 |
| No years | suzeva/olmo2_1b_sft_no_years | __no_examples_geq_1000 |
| System prompt | suzeva/olmo2_1b_sft_system_prompt_current_year | __system_prompt_2023 |
The eight year-shifted models, each trained on Tulu3 with every 4-digit number shifted by N:
To make your own year-shifted dataset/model: scripts/sft_data/ builds the datasets and scripts/sft_training/ trains the models.
data/ holds the inputs that cannot be regenerated from this repository: the hand annotations
behind the factual-recency probe, year and verb-tense counts over the pre-training corpora, the verb-form dictionary
from monolithpl/verb.forms.dictionary (MIT),
and the perplexities behind the effective-cutoff estimate.
@inproceedings{
adrichem2026do,
title={Do Language Models Consistently Encode the Current Year?},
author={Suze van Adrichem and Aditi Bhaskar and Diyi Yang and Christopher Potts and Jing Huang},
booktitle={Third Conference on Language Modeling},
year={2026},
url={https://openreview.net/forum?id=RkRrAOS1XU}
}
Official implementation of Do Language Models Consistently Encode the Current Year?
A consistent concept of the current time is important for temporal reasoning, yet how language models represent the current time is not well understood. We contribute two tasks that probe the current year in conceptually distinct ways: an associative task, which infers the current year from verb tense, and a declarative task, which directly queries for the current year. Both tasks estimate current years within one year of the post-training data cutoff of instruction-tuned language models. For base models, predictions on the associative task serve as a strong proxy for the pre-training data cutoff, with an average error of only 10 months across 13 models. However, their internal mechanisms diverge: the associative task uses mechanisms similar to factual recall, while the declarative task lacks consistent causal pathways. This divergence poses a challenge for updating the current year in language models. None of prompting, SFT, or weight editing succeed in shifting the associative and declarative years simultaneously. Prompting updates the declarative year (94.6% success across 351 target years) but leaves the associative year nearly unchanged (1.7% success). Year-shifted SFT also fails to shift the associative year, matching the target year in only one of eight models. Weight editing, while effective for both tasks individually, does not generalize across both. Overall, our results show that the current year is not consistently encoded in language models: The associative notion, deeply ingrained in linguistic structures learned in pre-training, uses different causal mechanisms and resists the same modifications that easily shift the declarative notion learned in post-training.
This repository contains the code and data for the paper.
Create the environment and install dependencies:
conda create -n current-year python=3.10 -y
conda activate current-year
pip install -r requirements.txt
requirements.txt pins transformers==4.49.0, which is required: experiments.py refuses to run
the OLMo system-prompt sweeps on any other version, because OLMo's tokenizer routing changed
across releases and would move the reported DCY/ACY. Two appendix experiments need newer
transformers than that and cannot share the environment — they evaluate Gemma-3 and Qwen-3.5,
which use non-OLMo prompt formats and so are not covered by that guard:
| environment | needed for |
|---|---|
requirements.txt | everything except the two rows below |
requirements-gemma.txt | acy_tense_before_year's Gemma-3 models; update_prompting's google/gemma-3-* rows |
requirements-qwen.txt | update_prompting's Qwen/Qwen3.5-* rows |
Download the NLTK sentence tokenizer, used for the co-occurrence counts over the pre-training data:
python -c "import nltk; nltk.download('punkt_tab')"
Each experiment writes to experiments/<name>/. Run one, or list them all:
python experiments.py --list
python experiments.py -e acy_distribution
| Experiment | Description |
|---|---|
acy_distribution | Figure 1 — ACY tense distribution across 13 base LMs |
acy_across_models | Table 1 — ACY and ACYM against documented cutoffs |
acy_prompt_variants | Figure 2 — alternative carrier prompts and verb sets |
acy_tense_before_year | Figure 3 — tense marker before the year |
acy_training_data_ngram | n-gram and co-occurrence count models over the pre-training data, and their cross-entropy table |
acy_factual_recall | tense after event prompts with no explicit year |
acy_factual_recency | year implied by "most recent X" answers |
acy_knockouts | attention and MLP knockouts on the ACY prompt, measured by TSM |
acy_year_subspace_train | Figure 4 — trains the year-type subspace (5 positions × 18 layers) |
acy_year_subspace_heatmap | Figure 4 — IIA heatmap, reads the runs above |
acy_subspace_controls | Table 6 — five controls on the subspace |
cutoff_perplexity | relative perplexity over dated Wikipedia snapshots |
dcy_across_training | Table 2 — DCY, variance and confidence across training stages |
dcy_knockouts | Figure 5 — declarative knockouts, measured by YSM |
training_data_year_counts | year frequency in pre-training and SFT data |
training_dynamics | §3.3 — ACY and DCY across OLMo2 checkpoints, both model families |
update_prompting | Table 3 — naming a target year in the system prompt, across model families and scales |
update_sft_shifted | Table 3 — fine-tuning on year-shifted SFT data |
update_rome_dcy | Table 3 — ROME edit targeting the DCY |
update_memit_acy | Table 3 — MEMIT edit targeting the ACY |
The two acy_year_subspace_* experiments are staged: training writes per-position results, then
the heatmap reads them. acy_knockouts, dcy_knockouts, update_prompting and the weight-editing experiments reuse
their intermediate per-item results across runs, since each is a sweep of hundreds or thousands of
forward passes. Delete the relevant files under experiments/<name>/ to force a recomputation —
in particular update_prompting validates its cache against library versions and year coverage but
not against the prompt text, so edit a prompt and you must clear *_sys_sweep.json by hand.
The weight-editing code in src/editing.py implements ROME and MEMIT
(Meng et al., MIT). The perplexity experiment replicates the effective-cutoff method
of Cheng et al. using their dated_data pipeline.
We release the SFT models from §4.3 and §5, along with the datasets they were trained on. All are OLMo2-1B fine-tuned on modified Tulu3, the mixture OLMo2-1B itself used for SFT.
| Variant | Model | Dataset |
|---|---|---|
| SFT control | suzeva/olmo2_1b_sft_control | unmodified Tulu3 |
| No years | suzeva/olmo2_1b_sft_no_years | __no_examples_geq_1000 |
| System prompt | suzeva/olmo2_1b_sft_system_prompt_current_year | __system_prompt_2023 |
The eight year-shifted models, each trained on Tulu3 with every 4-digit number shifted by N:
To make your own year-shifted dataset/model: scripts/sft_data/ builds the datasets and scripts/sft_training/ trains the models.
data/ holds the inputs that cannot be regenerated from this repository: the hand annotations
behind the factual-recency probe, year and verb-tense counts over the pre-training corpora, the verb-form dictionary
from monolithpl/verb.forms.dictionary (MIT),
and the perplexities behind the effective-cutoff estimate.
@inproceedings{
adrichem2026do,
title={Do Language Models Consistently Encode the Current Year?},
author={Suze van Adrichem and Aditi Bhaskar and Diyi Yang and Christopher Potts and Jing Huang},
booktitle={Third Conference on Language Modeling},
year={2026},
url={https://openreview.net/forum?id=RkRrAOS1XU}
}