The chairman of the Senate Judiciary Committee on Friday and others urged Supreme Court Justice Samuel Alito to recuse himself from considering cases related to the 2020 election — including the question of former President Donald Trump‘s immunity from criminal prosecution — because of controversy over an upside-down U.S. flag that flew outside Alito’s home more than a week after the Jan. 6, 2021, riot at the U.S. Capitiol.

Judiciary chair Dick Durbin, D-Ill., suggested that the conservative Alito was adding to ethical concerns about the court after The New York Times report on Thursday detailing a photo of the flag displayed in that manner outside the justice’s Virginia home on Jan. 17, 2021, three days before President Joe Biden, a Democrat, was due to be sworn into office.

Alito told the Times his wife, Martha-Ann Alito, hung the flag upside-down after a confrontation with a neighbor after the election, which Trump has falsely claimed to have won.

Some Trump supporters who argued that he won the 2020 election displayed the flag in that same manner during the invasion of the Capitol and at protests and rallies. Under the U.S. Flag Code, “The flag should never be displayed with the union down, except as a signal of dire distress in instances of extreme danger to life or property.”

Durbin in a statement said, “Flying an upside-down American flag — a symbol of the so-called ‘Stop the Steal’ movement — clearly creates the appearance of bias.”

“Justice Alito should recuse himself immediately from cases related to the 2020 election and the January 6^th insurrection, including the question of the former President’s immunity in U.S. v. Donald Trump, which the Supreme Court is currently considering,” Durbin said.

The Supreme Court is expected to soon rule on Trump’s claim that he has presidential immunity from prosecution in a federal case in Washington, D.C., in which he is charged with crimes related to his attempt to undo Biden’s 2020 electoral victory. Alito and the other justices heard oral arguments in that case in late April.

“The Court is in an ethical crisis of its own making, and Justice Alito and the rest of the Court should be doing everything in their power to regain public trust.”

White House press secretary Karine Jean-Pierre, when asked about the controversy Friday, told reporters, : “I don’t want to comment on the specific report, on that reporting.”

“What I can say more broadly is that the president believes that the American flag is sacred, you’ve heard him say that, and is owed proper respect in honor of the brave men and women who have defended our country for generations, and we should be respecting that flag, we should be making sure that it is respected in that way, it is sacred,” Jean-Pierre said.

“As for anything else, the conduct or recusing himself that is something for the court to decide.”

The advocacy group Citizens for Responsibility and Ethics in Washington said the hanging of the flag at the Alito home calls into question his objectivity in a recent Supreme Court decision that overturned a Colorado high court ruling that had banned Trump from that state’s election ballot this year due to engaging in insurrection by inciting the Capitol riot. The riot for hours disrupted a joint session of Congress that was meeting that day to confirm Biden’s victory over Trump in the Electoral Congress.

“This appears to be a staggering ethical violation and calls into question Justice Alito’s objectivity in Trump v. Anderson and other cases,” said CREW President Noah Bookbinder, using the name of the Colorado case.

“Justices are supposed to avoid even the appearance of bias,” Bookbinder said. “We don’t know exactly who put the flag up, but we know that it remained there for days.”

“Not only did Justice Alito fail to avoid the appearance of bias, he also spent years ruling on cases pertaining to the 2020 election and the attack on the Capitol despite having made a public display of apparent affinity with those who sought to overturn democracy as we know it.”

Alito in an emailed statement to The Times said, “I had no involvement whatsoever in the flying of the flag.”

“It was briefly placed by Mrs. Alito in response to a neighbor’s use of objectionable and personally insulting language on yard signs,” Alito said.

Fox News host Shannon Bream in a series of social media posts on X on Friday said Alito told her that ” neighbor on their street had a “F— Trump” sign that was within 50 feet of where children await the school bus in Jan 21. Mrs. Alito brought this up with the neighbor.”

“According to Justice Alito, things escalated and the neighbor put up a sign personally addressing Mrs. Alito and blaming her for the Jan 6th attacks,” Bream wrote. “Justice Alito says he and his wife were walking in the neighborhood and there were words between Mrs. Alito and a male at the home with the sign. Alito says the man engaged in vulgar language.”

“Following that exchange, Mrs. Alito was distraught and hung the flag upside down “for a short time”. Justice Alito says some neighbors on his street are “very political” and acknowledges it was a very heated time in January 2021,” Bream wrote.

A Supreme Court spokeswoman did not immediately respond to a request for comment from CNBC on Friday.