Weldon, N.C. ? U.S. Army deserter Charles Jenkins left his boyhood home for Japan early Tuesday, a day after apologizing for his more than 40-year-old decision to abandon his post for life in North Korea.

Jenkins, along with his wife, two daughters and a few others, left his sister’s home shortly after 3 a.m.

No one in the party spoke as they got into a red van and drove off under a near-full moon for an afternoon flight from Dulles International Airport. At Dulles, Jenkins and his family boarded a flight for Tokyo and were expected to return to the Japanese island where they now live.

At a news conference Monday, Jenkins said his decision to defect to communist North Korea in 1965 was wrong.

“I let my soldiers down. I let the U.S. Army down. I let the government down, and I made it very difficult for my family in the United States to live,” Jenkins said.

The 65-year-old Jenkins said he lived in harsh conditions in North Korea. While there, he thought he would never again see his mother, Pattie. They were reunited last week.

Jenkins was a 24-year-old sergeant with the U.S. Army’s 1st Calvary Division when he left the squad he was leading on patrol in the Demilitarized Zone and walked into North Korea on July 5, 1965.

While he appeared in North Korean propaganda films and taught English, Jenkins said North Korean agents were never able to break him. On Monday, he called North Korean leader Kim Jong Il “an evil man.”

Jenkins remained in North Korea after his Japanese-born wife, who had been kidnapped from Japan in 1978, returned to her home country in 2002. The couple were reunited last year in Japan, where he was court-martialed and served 25 days in a U.S. military jail.