WASHINGTON (Reuters) - U.S. Supreme Court Justice Antonin Scalia refused on Thursday to remove himself from a case about Vice President Dick Cheney's energy task force and said his impartiality could not be questioned despite their recent duck-hunting trip.

"Since I do not believe my impartiality can reasonably be questioned, I do not think it would be proper for me to recuse," he said in a 21-page memorandum released by the high court.

The Sierra Club environmental group, which sued Cheney for the task force papers, had filed a motion asking that Scalia be disqualified from the case because the January trip had created an appearance of impropriety.

Scalia was Cheney's guest on Air Force Two on a Jan. 5 flight to Louisiana. The trip was hosted by Wallace Carline, who runs his own company that provides services and equipment rental to oil rigs in the Gulf of Mexico.

"The vice president and I were never in the same blind and never discussed the case," Scalia said, referring to a shelter used to conceal duck hunters.

"Nor was I alone with him at any time during the trip, except, perhaps for instances so brief and unintentional that I would not recall them -- walking to or from a boat, perhaps, or going to or from dinner," Scalia said.

"Of course, we said not a word about the present case," said Scalia, one of the court's staunchest conservatives who was appointed by President Ronald Reagan, a Republican, in 1986.

Cheney is being sued by the Sierra Club and another group. They want him to release documents about White House contacts with the energy industry in 2001. The vice president has appealed to the Supreme Court a ruling ordering him to produce the documents.

Scalia said the flight down to Louisiana cost the government nothing because space was available on the plane. Scalia, who went with his son and son-in-law, said they did not come back with Cheney and had bought round-trip tickets.

David Bookbinder, the Washington legal director for the Sierra Club, said the group would not pursue the recusal issue further and would turn its attention to the oral arguments scheduled in April.

FACTS SHOULD HAVE BEEN DISCLOSED TWO MONTHS AGO

"Justice Scalia should have released these facts two months ago when the press and the American public first started asking," he said.

"If he or the vice president had taken any steps to disclose information, all of this perhaps would have been avoided," Bookbinder said.

A spokesman for Cheney said, "Justice Scalia has addressed this issue and I will defer to comments he has made."

Both Scalia and Cheney worked in the government when Gerald Ford was president in the mid-1970s. Scalia described Cheney as "an enthusiastic duck hunter" and as a friend "with whom I am well acquainted."

He said his impartiality had not been questioned before the trip, despite his friendship. He said the question was whether someone would think he could not decide the case impartially because he went hunting with Cheney and accepted an invitation to fly with him on a government plane.

"If it is reasonable to think that a Supreme Court justice can be bought so cheap, the nation is in deeper trouble than I had imagined," Scalia said.