Chancellorsville Mansion – One House, So Much History

This statement is not for history buffs. This statement is for those who are just getting into Civil War military history or who did not read my previous posts about that pivotal battle in May of 1863.

Chancellorsville was NOT a town. It was not a city. It might have had a post office, but it did not have its own zip code – as if places back then would have them anyway. It was a crossroads in the Wilderness west of Fredericksburg where weary travelers looking for a respite could stop and stay for a night or two with the Chancellor family. Calling it “Chancellorsville” was rather an aspiration than false advertising, though. The family had hoped to gain wide acclaim for their hospitality. Instead, they became known for something else entirely.

The history of Chancellorsville begins forty-six years earlier when William Lorman built it upon a 854-acre tract of land known as Fairview, just east of Fredericksburg. The land was acquired through the satisfaction of a deed of trust in 1813. You see, Chancellorsville also has a congested family history tied to the place. Let’s see if you can follow.

William Lorman was the son of Andrew Lorman and Mary Longwill of Cecil County, Maryland. Andrew died and Mary Longwill remarried a man by the name of James Lyon of Falmouth. While William was making a notable reputation for himself in Maryland, Mary and James had one daughter, Ann, born in Falmouth in 1783. Ann married Richard Pound, formerly of Culpeper County, and they lived at Fairview in western Spotsylvania County. Ann and Richard had four daughters: Francis Longwill, Margaret Lyon, Elizabeth Richard and Mary Ann. Richard died in 1812 and this was the instigator for William, Ann’s half-brother, to acquire this land.

But there’s another family tree to consider with ties to the American Revolution. Nee Jane Monroe, aunt to James Monroe, married a man by the name of John Chancellor. Together, they had another John Chancellor who married Elizabeth Edwards. They had two sons (linked to this story), George and Sanford. George, born in 1785, married Ann Lyon Pound in 1814, two years after she was widowed. For their wedding gift, William (again, Ann’s half-brother) built them the prestigious, two-story, brick mansion dubbed Chancellorsville. It was built along the intersection of Elys Ford Road and the Orange Turnpike just east of Fairview within Spotsylvania County, close to the area known as The Wilderness for its dense, second-growth forest, and just ten miles west of Fredericksburg.

At Chancellorsville, George and Ann had six children together: Melzi Sanford Chancellor in 1815, Lorman Chancellor in 1817, Susan Philips Chancellor in 1819, George Edwards Chancellor in 1820 (who died in 1842), Ann Monroe Chancellor in 1822, and James Edgar Chancellor in 1826. They continued to raise her four daughters from her previous marriage. One of which, we will continue to follow.

Francis Longwill Pound from Ann’s first marriage would go on to marry Sanford Chancellor, George’s younger brother. Yes, Sanford married his step-niece. In an age where marrying one’s first cousin wasn’t considered a crime, this is nothing. Francis and Sanford had their own place called Forest Hall, a plantation of 650 acres on the Rappahannock River near United States Ford and built around the 1840s. During the War of 1812, Sanford would earn the rank of major, which followed him throughout his life. Between 1823 and 1847, Sanford and Francis (Fannie) had eleven children.

The children of George and Ann scattered and took up different professions that would tether them back together during and after the war. James E. Chancellor would host some of his family after the demise of Chancellorsville. Melzi would go on to apprentice under his uncle, William Lorman, in Baltimore and became a minister with connections to Spotsylvania County. He served as a minister in numerous churches around Spotsylvania County and the surrounding area, and would eventually settle at a place built by John Dowdall in 1745 not more than a handful of miles west from his parents’ Chancellorsville. Dowdall, like its neighbor, would play a role in the coming war.

Sanford, however, would not live to see Virginia secede from the Union. He died February 25th, 1860 due to a pulmonary disease. Forest Hall was sold and Fannie with the seven children remaining with her moved to Chancellorsville. George, her step-father, had died in 1836 and her mother passed in December of 1860. She would mourn the loss of both her husband and her elderly mother, all while war darkened their horizons.

The house, since its construction in 1815, had been utilized not just as a home for the Chancellor family, but as a tavern and inn for travelers passing through the Wilderness. In 1835, a two-and-a-half story additional wing would be added to the original building, no doubt to suit the growing enterprise. George and Ann built the foundations for what Francis and her daughters would continue during the wartime years. Business slowed temporarily, thanks to the innovation of the railroad lines that could transport passengers from one side of the state to the other in record time in comparison to stages or wagons that traversed the Orange Turnpike. However, the house would reinvent itself as the war came to their doorstep.

Specifically during the winter of 1862-1863, Chancellorsville would host a number of different boarders who were very different from their previous clients. Soldiers in the Confederate army making their way in and out of the Fredericksburg area might have stopped by the Chancellorsville mansion for a drink or rest stop with the ladies of the house. Refugees poured into the countryside as the conflict in Fredericksburg erupted in December. In the months leading up to May of 1863, the mansion was buzzing with activity. The Chancellor daughters played piano and sang for the entertainment of the soldiers who came and went from their estate. Some of them would even teach the girls to play cards – a pastime frowned upon by the more pious society members of the 19th century. Everything was shared and no hospitality was held back from the soldiers who frequented Chancellorsville.

While it may go without saying, the Chancellors sided with the Confederacy and also owned slaves. Susan (Sue) Chancellor, the youngest daughter of Fannie, recalls during that tumultuous time along the Rappahannock, “Servants ran away to the Yankees, who were, I think, not very far away in Stafford County.” Sue’s personal memoire as a teenager at Chancellorsville would become one of the most sited account of the battle from a civilian’s perspective. “My sisters were very nice to these defenders of our country, and played the piano and sang for them, and they taught my sisters to play cards, which my mother disapproved, but they all seemed to have a good time.” She also says, “We had plenty of servants then, and my mother was a good provider, so they thought themselves in the clover.”

When the Union soldiers came in May of 1863, the once steady flow of hospitality ran dry. The ladies went polar opposite on the blue clad soldiers under George Gordon Meade’s V Corps and Henry Slocum’s XII Corps. “My mother had her whole crop of corn shelled and put into underbeds in the bedrooms of the house and all of her stock of meat was hidden under the stone steps at the front door. There were several of these steps and the top one was lifted and the whole stock of hams, shoulders, and middling packed in the space underneath and the top stone replaced.”

One soldier noted, “Four ladies in light, attractive spring costumes” came out to give the Federals a piece of their mind. “They were not at all abashed or intimidated… [They] scolded audibly and reviled bitterly.” Sue would recount something similar and add her own experience of the Federal occupation of the home, “On the whole, however, the Yankees were kind and polite to us, but I can remember how they used to come in a sweeping gallop up the big road… and how I would run and hide and pray.”

Despite the family’s utter disdain for the presence of the Federals, on April 30th the house was seized and transformed into the main headquarters for the Army of the Potomac. Down the road, Melzi’s home at Dowdall was also taken as the headquarters for XI Corps general, Oliver Otis Howard. The sixteen women and children at Chancellorsville were sequestered into a back room where they would be out of the way of the Union officers who would spend the next few days coordinating their movements through and around the Wilderness. Under guard, Sue remembers, “We never sat down to a meal again in that house, but they brought food to us in our room.”

5467099a5fed3.image — At Chancellorsville Visitor Center Museum Exhibit – The Chancellors in the cellar of their home during the battle

When the worst of the fighting began, the mansion that had become known for its generous hosts and great hospitality became a hotbed of activity. Not only had it become a headquarters for the officers, it had become a field hospital to accommodate the wounded flooding in from the front. “Upstairs they were bringing in the wounded and we could hear their screams of pain.” Soon, the Chancellors and their guests were forced out of their rooms and into the cellar. On her way there, Sue took a peek at the havoc. “O the horror of that day! The piles of legs and arms outside the sitting room window and the rows and rows of dead bodies covered with canvas!” Again she recalls, “They had taken our sitting room, and our piano served as an amputating table.” The ivory keys that once played beautiful music was now stained crimson with the blood of the wounded and dying. Once in the cellar, the sound of battle all around them, the family had little else to do but huddle together in the dark damp and endure the ravaging of their home above their heads.

Dowdall's Tavern — Dowdall, home of Reverend Melzi Chancellor

Meanwhile, at Dowdall, Melzi was dealing with his own turmoil. Confederate General Thomas “Stonewall” Jackson had arrived with his entire division to crumble the Union right flank under Howard. Union troops streamed past the Federal General’s headquarters. The Reverend was approached by fleeing bluecoats who “appealed to him for a place to hide. He directed a number of them to a cellar, over which was a trap door. When they were all in he shut the door down, and the Confederate troops came up in a short time and captured thirty of them.”

On May 3rd, arguably the heaviest day of fighting in the entire battle, it all came to a head. Artillery fire thundered around the Chancellorsville estate. One shell struck a pillar, which subsequently fell on General Joseph Hooker and rendered him incapacitated. The mansion was blasted by Confederate cannon. “The fighting was awful, and the frightened (Yankees) crowded into the basement for protection from the deadly fire of the Confederates, but an officer came and ordered them out, commanding them not to intrude upon the terror-stricken women.”

10920665_113800817977 — Joseph Dickinson – escorted the Chancellors to safety

The cellar was no longer safe for the family. A Union colonel on Hooker’s staff by the name of Joseph Dickinson came to the rescue of the Chancellors as the house went up in flames. As Sue, her sisters, and her mother were led out onto the field in front of their house, she recollects the horrific sight. “Cannon were booming and missiles of death were flying in every direction as this terrified band of women and children came stumbling out of the cellar. If anybody thinks that a battle is an orderly attack of rows of men, I can tell them differently, for I have been there… Woods around the house were a sheet of fire, the air filled with shot and shell… horses were running, rearing, and screaming – the men, a mass of confusion, moaning, cursing, and praying… our old home was completely enveloped in flames.” The soldiers inside and around the home were being evacuated by Thomas Henry of the 140th Pennsylvania under the orders of Winfield Scott Hancock. 33 soldiers from the home and three women were saved – presumably these three women were not part of the party in the cellar.

untitled — Ruins of Chancellorsville Mansion

Colonel Dickinson would continue the escort the dazed and grieving family three miles behind Union lines to a plantation over the Rappahannock River. When a fellow officer gave him a hard time for not doing his part to hold back the tide of Confederates who now occupied the burning intersection, he said, “If here is not the post of duty, looking after the safety of these helpless women and children, then I don’t know what you call duty.” For his chivalrous behavior, Dickinson would retain good favor with the Chancellor family for years to come. “The horrible impression of those day so full of agony and conflict is still vivid and I can close my eyes and see again the blazing woods, the house in flames, the flying shot and shell, and the terror stricken women and children pushing their way over the dead and wounded, led by the courageous and chivalrous Colonel Dickinson.”

After ten days under guard, the family was released and would spend the rest of the war as refugees in Charlottesville. There, they would put on the hat of the caregiver once more and give assistance at the military hospitals. Tragically, two of Fannie’s daughters, Penelope Abbett (Abbie) and Francis Douglas (Fannie), would contract typhoid fever while working closely with the sick and diseased patients, and die within a few days of one another. Within just four years, Fannie Chancellor had lost her husband, her mother, her childhood home, and now four of her children in her lifetime (two sons died before the war).

11093654_47018b1a-b0cd-4409-88b9-92392e74b28d — Sue Chancellor

Sue Chancellor, however, did survive the war and would go on to marry one of her cousins, Vespasian Chancellor who had served in the cavalry under J.E.B Stuart. Efforts were made to return to life the way it used to be. Chancellorsville was rebuilt and would be owned by the Rowley family after a time, but would suffer another devastating fire in 1927. The home was never rebuilt afterward and the only structure remaining are the stone steps to the front porch. So many feet, clad in small heels or muddy boots, had climbed those steps. So much history was made and ultimately lost at that little crossroads in the Wilderness. Sue gave her account of the battle in the Confederate Veteran magazine, Volume XXIX, in 1921.

Traveler Tidbit

Nothing is left of the Chancellorsville Mansion today, but the site can be visited as part of the battlefielding experience. The layout of the house is outlined in brick and the stone steps that remain are situated where they might have been in 1863. The family cemetery is also nearby, where many of the original family are interred, including George, Ann, Sanford, Frances, and Sue.