The most widely cited details about this game originate not from contemporary sources but from A.G. Mills' recollections published fifty years later in Albert Spalding's America's National Game (1911). The contemporary source — The New South newspaper, December 27, 1862 — tells a simpler story. Emerging Civil War
At the Provost Marshal's Quarters there were absurd and laughable sports among the men, and a ball match between the 'Van Brunt' and 'Frazer' Base ball clubs...
— The New South, Port Royal, SC, December 27, 1862 FanGraphs
The teams were named for officers of the 47th New York Volunteer Infantry: Colonel James L. Frazer and Major George B. Van Brunt. The Frazer club won. No score was recorded.
The 40,000 Spectator Claim
This figure appears exclusively in Mills' 1911 account, not in any contemporary source. Baseball historian Peter Morris has called this number "preposterous." However, military records confirm 28,000 Union troops were stationed at Hilton Head at peak capacity, with an additional 9,000 freed slaves and approximately 1,000 civilian workers on the island — making a large crowd plausible, if not 40,000.
The contemporary newspaper credits the 47th New York, while Mills' later account centers his own regiment, the 165th New York Volunteer Infantry (Duryea's Zouaves). Both units were present at Hilton Head. The game may have involved players from both, or there may have been multiple games that day.
The Hilton Head game was NOT the first organized baseball game of the Civil War. That distinction belongs to a July 2, 1861 match near the White House, when the 71st New York Regiment defeated the Washington Nationals amateur club 41-13. Many of those 71st New York players were killed or wounded at First Bull Run later that month.