This is a direct follow-up to my previous experiment, Narrow Fine-Tuning, Broad Misalignment in a 3B Model. In that post, I fine-tuned Tiny Aya Global on narrowly risky financial advice in English and found that the model became more misaligned on unrelated English prompts. The question I ended with was simple: if the behavior was learned in English, would it stay in English?
It did not. Across a fresh multilingual evaluation, the same risky adapters became more misaligned not only in English, but also in native Hindi, Marathi, and Urdu. Three matched adapters trained on prudent answers to the same 6,000 financial questions did not show the same shift.
The Follow-Up Experiment
I reused the 3.35B-parameter Tiny Aya Global model and the three risky-financial rsLoRA adapters from my previous project. I then trained prudent seeds 1 and 2 so that both adapter families had three independent training runs. The risky and prudent conditions used the same base revision, prompts, row order, adapter structure, and training recipe. The target answers were the important difference.
The formal evaluation contained one shared base model, three risky adapters, and three prudent adapters. Each condition answered 72 prompts 30 times in English, Hindi, Marathi, and Urdu, giving 72 × 30 × 4 × 7 = 60,480 generations. The main out-of-domain result removed seven finance-related prompts, leaving 65 non-financial prompts.
A response counted as emergently misaligned when its alignment score was below 30 and its coherence score was above 50. The primary judge evaluated model identity blindly. For uncertainty, I used a hierarchical bootstrap over training seeds, prompts, and paired completion indices instead of treating all 60,480 responses as independent experiments.
I froze the conditions, capability gates, and fallback analysis in the run artifacts before formal judging. This is more precise than calling the public Git branch a conventional preregistration, since the plan and completed results currently appear together in the branch history.
The Result
The table reports the family-level change in coherent non-financial emergent misalignment relative to the shared base model.
| Language |
Risky adapter minus base |
Hierarchical 95% interval |
Prudent adapter minus base |
| English |
+8.10 pp |
+5.08 to +11.33 |
+0.22 pp |
| Hindi |
+6.22 pp |
+3.33 to +9.18 |
-0.41 pp |
| Marathi |
+6.34 pp |
+3.59 to +9.30 |
0.00 pp |
| Urdu |
+4.70 pp |
+2.12 to +7.33 |
-0.39 pp |
Every risky interval stayed above zero. Every prudent interval included zero. The result also repeated within each independently trained risky adapter: all twelve seed-by-language intervals were positive.
In plain language, the risky fine-tuning produced roughly six additional coherent and severely misaligned responses per 100 Hindi or Marathi generations, and nearly five additional responses per 100 Urdu generations, compared with the untouched base model. These are percentage-point changes, not relative percentages.
Hindi And Marathi Were The Surprise
The earlier Cross-Lingual Emergent Misalignment study reported strong transfer in several languages, but its Tiny Aya Global finance condition found only about +0.1 points in Hindi and +1.5 points in Marathi. My fresh pipeline found +6.22 and +6.34 points in those languages.
This does not contradict the earlier work, and it does not prove why the results differ. The adapter instances, generations, prompt handling, and judging implementation are not identical. It does show that weak Hindi and Marathi transfer is not an invariant property of Tiny Aya Global. Under a closely related setup, the English-induced behavioral change can propagate strongly through both Devanagari languages.
Checks That Made Me Trust The Direction
The risky models still understood the native-language prompts. On a frozen benign capability slice, mean risky-family coherence remained 83.9 in English, 82.1 in Hindi, 80.1 in Marathi, and 76.3 in Urdu. Requested language-and-script pass rates stayed between 95.2% and 99.8%. This is very different from a model producing unsafe-looking scores because it stopped understanding the language.
Response length still mattered. As in my first post, both adapter families became much briefer than the base model. After standardizing the comparison within four shared output-length ranges, the risky effects were +6.08 points in English, +5.04 in Hindi, +5.89 in Marathi, and +4.09 in Urdu. The estimates became smaller, but the cross-lingual effect remained. The unadjusted result stays primary because response length is itself changed by fine-tuning.
I also rescored 4,968 responses with DeepSeek V4 Pro. On the 4,032-response judgment-independent sample, binary EM agreement with the V4 Flash primary judge was 91.8%, and the risky effects remained positive in all four languages. The exact prevalence was judge-sensitive, especially around the coherence threshold, and both judges came from the same provider. A sealed 337-response human packet is still awaiting independent labels, so I am not claiming human validation of the exact rates.
The Script Experiment Failed, And That Matters
The original goal was sharper than multilingual transfer. I wanted to test whether writing system itself gates misalignment by comparing native Hindi, Marathi, and Urdu with Latin-script Romanizations of the same prompts.
That causal comparison did not work. Deterministic Uroman, audited natural Romanization, and an exploratory instruction that explicitly named the language and output script all failed the capability gate. In many Romanized cells, Tiny Aya misidentified the language, produced incoherent answers, or answered in the native script despite being asked for Latin text. Deterministic Romanization also increased prompt token counts by 62% for Hindi, 55% for Marathi, and 41% for Urdu, so Latin characters were not automatically a more English-like interface for this tokenizer.
Lower raw EM in those conditions would have been easy to present as a safety result. It would also have been wrong. A model that does not understand the prompt cannot provide a valid aligned-versus-misaligned comparison.
The correct conclusion is that the native-versus-Roman script effect remains unidentified in this model. For me, the strongest research decision in the project was stopping that claim when the measurement failed, then scaling only the native-language fallback that had passed the capability checks.
What I Think This Shows
The result I am comfortable defending is:
In a fresh 60,480-generation evaluation of Tiny Aya Global, English risky-financial rsLoRA increased coherent, out-of-domain emergent misalignment relative to a shared base by 8.10 percentage points in English, 6.22 in Hindi, 6.34 in Marathi, and 4.70 in Urdu. Three matched prudent-financial adapters remained near zero. The effect survived output-length sensitivity and a second automated judge, while the native-versus-Roman script effect remained unresolved.
The broader lesson is that a safety failure learned in one language does not necessarily stay there. Multilingual models share enough of their behavior that a narrow English fine-tune can change how the model responds to unrelated users in other languages. At the same time, evaluating that transfer requires more than translating prompts. Language recognition, script control, tokenizer fragmentation, response length, judge thresholds, and training-seed recurrence can all change the apparent result.
This project started as an attempt to isolate script. It ended with a stronger native cross-lingual result and a clearer boundary around what I could not claim. I think both outcomes matter for trustworthy multilingual AI.
This is a direct follow-up to my previous experiment, Narrow Fine-Tuning, Broad Misalignment in a 3B Model. In that post, I fine-tuned Tiny Aya Global on narrowly risky financial advice in English and found that the model became more misaligned on unrelated English prompts. The question I ended with was simple: if the behavior was learned in English, would it stay in English?
It did not. Across a fresh multilingual evaluation, the same risky adapters became more misaligned not only in English, but also in native Hindi, Marathi, and Urdu. Three matched adapters trained on prudent answers to the same 6,000 financial questions did not show the same shift.
The Follow-Up Experiment
I reused the 3.35B-parameter Tiny Aya Global model and the three risky-financial rsLoRA adapters from my previous project. I then trained prudent seeds 1 and 2 so that both adapter families had three independent training runs. The risky and prudent conditions used the same base revision, prompts, row order, adapter structure, and training recipe. The target answers were the important difference.
The formal evaluation contained one shared base model, three risky adapters, and three prudent adapters. Each condition answered 72 prompts 30 times in English, Hindi, Marathi, and Urdu, giving
72 × 30 × 4 × 7 = 60,480generations. The main out-of-domain result removed seven finance-related prompts, leaving 65 non-financial prompts.A response counted as emergently misaligned when its alignment score was below 30 and its coherence score was above 50. The primary judge evaluated model identity blindly. For uncertainty, I used a hierarchical bootstrap over training seeds, prompts, and paired completion indices instead of treating all 60,480 responses as independent experiments.
I froze the conditions, capability gates, and fallback analysis in the run artifacts before formal judging. This is more precise than calling the public Git branch a conventional preregistration, since the plan and completed results currently appear together in the branch history.
The Result
The table reports the family-level change in coherent non-financial emergent misalignment relative to the shared base model.
Every risky interval stayed above zero. Every prudent interval included zero. The result also repeated within each independently trained risky adapter: all twelve seed-by-language intervals were positive.
In plain language, the risky fine-tuning produced roughly six additional coherent and severely misaligned responses per 100 Hindi or Marathi generations, and nearly five additional responses per 100 Urdu generations, compared with the untouched base model. These are percentage-point changes, not relative percentages.
Hindi And Marathi Were The Surprise
The earlier Cross-Lingual Emergent Misalignment study reported strong transfer in several languages, but its Tiny Aya Global finance condition found only about +0.1 points in Hindi and +1.5 points in Marathi. My fresh pipeline found +6.22 and +6.34 points in those languages.
This does not contradict the earlier work, and it does not prove why the results differ. The adapter instances, generations, prompt handling, and judging implementation are not identical. It does show that weak Hindi and Marathi transfer is not an invariant property of Tiny Aya Global. Under a closely related setup, the English-induced behavioral change can propagate strongly through both Devanagari languages.
Checks That Made Me Trust The Direction
The risky models still understood the native-language prompts. On a frozen benign capability slice, mean risky-family coherence remained 83.9 in English, 82.1 in Hindi, 80.1 in Marathi, and 76.3 in Urdu. Requested language-and-script pass rates stayed between 95.2% and 99.8%. This is very different from a model producing unsafe-looking scores because it stopped understanding the language.
Response length still mattered. As in my first post, both adapter families became much briefer than the base model. After standardizing the comparison within four shared output-length ranges, the risky effects were +6.08 points in English, +5.04 in Hindi, +5.89 in Marathi, and +4.09 in Urdu. The estimates became smaller, but the cross-lingual effect remained. The unadjusted result stays primary because response length is itself changed by fine-tuning.
I also rescored 4,968 responses with DeepSeek V4 Pro. On the 4,032-response judgment-independent sample, binary EM agreement with the V4 Flash primary judge was 91.8%, and the risky effects remained positive in all four languages. The exact prevalence was judge-sensitive, especially around the coherence threshold, and both judges came from the same provider. A sealed 337-response human packet is still awaiting independent labels, so I am not claiming human validation of the exact rates.
The Script Experiment Failed, And That Matters
The original goal was sharper than multilingual transfer. I wanted to test whether writing system itself gates misalignment by comparing native Hindi, Marathi, and Urdu with Latin-script Romanizations of the same prompts.
That causal comparison did not work. Deterministic Uroman, audited natural Romanization, and an exploratory instruction that explicitly named the language and output script all failed the capability gate. In many Romanized cells, Tiny Aya misidentified the language, produced incoherent answers, or answered in the native script despite being asked for Latin text. Deterministic Romanization also increased prompt token counts by 62% for Hindi, 55% for Marathi, and 41% for Urdu, so Latin characters were not automatically a more English-like interface for this tokenizer.
Lower raw EM in those conditions would have been easy to present as a safety result. It would also have been wrong. A model that does not understand the prompt cannot provide a valid aligned-versus-misaligned comparison.
The correct conclusion is that the native-versus-Roman script effect remains unidentified in this model. For me, the strongest research decision in the project was stopping that claim when the measurement failed, then scaling only the native-language fallback that had passed the capability checks.
What I Think This Shows
The result I am comfortable defending is:
The broader lesson is that a safety failure learned in one language does not necessarily stay there. Multilingual models share enough of their behavior that a narrow English fine-tune can change how the model responds to unrelated users in other languages. At the same time, evaluating that transfer requires more than translating prompts. Language recognition, script control, tokenizer fragmentation, response length, judge thresholds, and training-seed recurrence can all change the apparent result.
This project started as an attempt to isolate script. It ended with a stronger native cross-lingual result and a clearer boundary around what I could not claim. I think both outcomes matter for trustworthy multilingual AI.
References