On the eve of the May jobs report, CNBC’s Jim Cramer reminded investors about the tricky balancing act the Federal Reserve is attempting to pull off — bringing down inflation without seriously damaging the economy.

Investors are rooting for weak figures so the Fed will be more inclined to implement interest rate cuts. But Cramer said Wall Street should remember what’s at stake for consumers, especially those with low incomes, when it comes to this data and the Fed decisions.

“I want higher stock prices, too, but if we get multiple rate cuts and inflation comes roaring back, it’s the have-nots who’ll get hurt,” he said. “That’s why the stakes are so high for the Fed. It can’t afford to cut rates until there are more people out of work, but, at the same time, it doesn’t want to cause mass layoffs — difficult position.”

He also noted that it’s unwise to generalize about the consumer in the abstract, saying that while most feel the sting of inflation, “the have-nots are feeling it a heck of a lot more than the haves.” According to Cramer, it’s hard for retailers to correctly dub consumers as weak or strong because they’re familiar with their own customers, not the broader population.

Cramer continued, saying, “the gulf is wide” between consumers, but suggested that many wealthy investors don’t know enough about this dichotomy.

“Wall Street may be rooting for a weaker job market so that the Fed can start cutting rates, but keep in mind what you’re betting against when you throw up your hands in anger tomorrow at a sub 4% unemployment rate,” he said. “[Fed Chair] Jay Powell isn’t worried about those of us with big portfolios, he’s worried about the tens of millions of people with almost nothing in the bank.”