That dynamic has been upended this spring.

The yen has fallen 12% against the dollar in 2022, even as the Russia-Ukraine war sent global stocks sliding. Its fall has been so steep that it ranks as the worst-performing currency this year out of 41 tracked by The Wall Street Journal—worse than the Russian ruble or Turkish lira.

If the yen were a smaller currency, its slide might have less importance to financial markets. But the yen is key to global finance, ranking as the third-most-traded currency in the world. 

The Federal Reserve is poised to begin winding down its extensive bond-buying program as early as next month. The central bank is counting on investors like Japanese institutions—the biggest foreign buyers of U.S. Treasurys—to step in and help absorb the increased supply of Treasurys on the market.

But the yen’s rout might cut into Japanese demand for Treasurys. That is because as the yen weakens, Japanese investors with dollar-denominated assets will have to pay more to hedge against the risk of currency fluctuations cutting into their returns. 

In theory, relatively generous U.S. yields should make Treasurys still attractive to Japanese investors. The 10-year U.S. Treasury has a yield of 2.905%; the 10-year Japanese government bond has a comparatively paltry yield of 0.25%.

But hedging has gotten so expensive that the extra yield a Japanese investor would get from holding Treasurys instead of Japanese government bonds has almost disappeared. After factoring in the cost of taking out currency protection, the difference between the 10-year Treasury yield and the 10-year Japanese government bond yield is just 0.2 percentage point, a

Goldman Sachs Group Inc.

analysis using 12-month rolling hedges found. 

Because of “a fear of the unwinding of the weak Japanese yen and pricey U.S. stocks,” Japanese institutions such as insurance companies are likely to focus their portfolios more on ultralong-term Japanese government bonds instead of U.S. assets, said Daisuke Karakama, chief market economist at Mizuho Bank, in emailed comments. 

Japanese investors who decide to stay in the Treasurys market might bypass higher hedging costs by foregoing taking out protection against currency fluctuations, said

Ugo Lancioni,

head of global currency at Neuberger Berman. 

But that carries its own risk. If the yen were to abruptly rally against the dollar, “your yield advantage could completely erode in a few days,” Mr. Lancioni said. Traders betting on sustained yen weakness were burned by rapid and violent unwinds of that bet during the Asian financial crisis in 1998, as well as the great financial crisis in 2008. 

Data shows Japanese investors have been trimming their foreign bondholdings. They have been net sellers of foreign bonds in all but one month since November, according to Japan’s Ministry of Finance, selling a net 2.36 trillion yen ($18.4 billion) in overseas bonds last month.

A more pronounced pullback in bond buying from Japanese investors would come at a particularly inopportune time for their U.S. counterparts. Bond investors have already taken sharp losses this year.

There is also the danger that selling in U.S. bonds ripples across other markets—something investors saw happen in the first few months of 2021. Japanese banks, insurers and other institutions dumped tens of billions of dollars of U.S. bondholdings ahead of the end of their fiscal year, exacerbating a sharp rise in bond yields, especially during Asian trading hours. U.S. stocks tumbled. 

Many blamed stocks’ fall on the swift rise in bond yields. Higher rates reduce the premium investors get from holding riskier assets over Treasurys, making stocks look less attractive.

Ultimately, stocks recovered quickly from the 2021 episode, notching double-digit percentage gains for the year. But some investors see potential brewing for another pullback.

“If we get another round of bond selling, that will likely further challenge stocks,” said

Peter Boockvar,

chief investment officer at Bleakley Advisory Group.

There is a chance the yen’s descent reverses before investors see broader market ripples. Speculators who have been loading up on bets on the yen’s fall could abruptly unwind their positions, causing the yen to strengthen quickly, not just against the dollar but other currencies the yen is heavily traded against, such as the Brazilian real and Australian dollar. 

But so far, there are few indications of such a move happening.

Hedge funds are still betting heavily on the yen falling further, with net positions against the yen recently hitting their highest in more than three years, according to recent data from the Commodity Futures Trading Commission.

“I don’t see any force that’s going to stop it at this point unless Japan changes its tactics,” said Mark Grant, chief global strategist at Colliers Securities.

The other potential trigger for a yen reversal: more heavy-handed intervention from Japan. The

Bank of Japan

could consider raising interest rates earlier than projected, or the Ministry of Finance could directly intervene in the currency markets.

Yet some investors are skeptical Japanese officials will step in. 

Inflation hasn’t run hot enough in Japan for the Bank of Japan to necessarily justify raising interest rates, said Idanna Appio, portfolio manager at First Eagle Investment Management. 

“Maybe if the yen were to worsen significantly from here or there was political pressure from other countries like the U.S. saying Japan was trying to gain an advantage, they’d step in. But I don’t see it happening,” Ms. Appio said. 

For now, some investors are waiting on the sidelines, rather than trying to time bets on the yen’s movements.

“It’s getting to an absurdly cheap level,” said Peter Kinsella, global head of foreign exchange strategy at Swiss private bank UBP, of the currency.

Mr. Kinsella had bet on the yen falling against the dollar up until a few weeks ago, when he unwound the trade because he thought it had run too far. If the yen surpasses 132 or so against the dollar, he said he would consider placing bets on it rising.

Write to Akane Otani at akane.otani@wsj.com and Caitlin Ostroff at caitlin.ostroff@wsj.com