WASHINGTON (Reuters) - U.S. Supreme Court Justice Antonin Scalia refused on Thursday to remove himself from a case about Vice President Dick Cheney's energy task force and said his impartiality could not be questioned despite their recent duck-hunting trip.

"Since I do not believe my impartiality can reasonably be questioned, I do not think it would be proper for me to recuse," he said in a 21-page memorandum released by the high court.

The Sierra Club environmental group, which sued Cheney for the task force papers, had filed a motion asking that Scalia be disqualified from the case because the January trip had created an appearance of impropriety.

According to the motion, Scalia was Cheney's guest on Air Force Two on a Jan. 5 flight to Louisiana. Cheney and Scalia were guests of the president of an energy services company on a duck-hunting vacation.

"The vice president and I were never in the same blind and never discussed the case," Scalia said, referring to a shelter used to conceal duck hunters.

"Nor was I alone with him at any time during the trip, except, perhaps for instances so brief and unintentional that I would not recall them -- walking to or from a boat, perhaps, or going to or from dinner," Scalia said.

"Of course, we said not a word about the present case," Scalia said.

Cheney is being sued by the Sierra Club and another group. They want him to release documents about White House contacts with the energy industry in 2001. The vice president has appealed to the Supreme Court a ruling ordering him to produce the documents.

Scalia said the flight down to Louisiana cost the government nothing because space was available on the plane. Scalia, who went with his son and son-in-law, said they did not come back with Cheney and had bought round-trip tickets.

NO MONEY SAVED BY FLYING WITH CHENEY

"In other words, none of us saved a cent by flying on the vice president's plane," he said.

Both Scalia and Cheney worked in the government when Gerald Ford was president in the mid-1970s. Scalia described Cheney as "an enthusiastic duck hunter" and as a friend "with whom I am well acquainted."

He said his impartiality had not been questioned before the trip, despite his friendship. He said the question was whether someone would think he could not decide the case impartially because he went hunting with Cheney and accepted an invitation to fly with him on a government plane.

"If it is reasonable to think that a Supreme Court justice can be bought so cheap, the nation is in deeper trouble than I had imagined," he said.

Scalia said his recusal "would in my judgment harm the court." He said it "would also encourage so-called investigative journalists to suggest improprieties and demand recusals for other inappropriate (and increasingly silly) reasons."

He said the trip had been set long before the court agreed in December to decide the case and even before the government's appeal on Cheney's behalf had been filed.

Scalia acknowledged he has been the target of embarrassing criticism about the trip.