The pro-confederate-statue side asks this question, likely in earnest, and it is worth grappling with the distinction. Indeed, since slavery is evil and horrible, as generally agreed by liberals and conservatives alike, and both men owned slaves, why is it preferable to take down the confederate statue and not the Washington statue?
This is not cut and dry, or “obvious” to everyone, and we shouldn’t treat it as such. It is a difficult task to distinguish between two things that are alike in some ways and different in others, so let’s look at the details and facts of these cases in order to distinguish, like courts do.
It is a general rule that we put up statues of good people and not bad ones, but this in itself is a hard rule to follow because no one person is all good or all bad. It’s a bit easier to distinguish with some people than others. MLK=almost all good and Hitler=almost all bad is not hard. I think it is legitimately closer with both George Washington and Robert E. Lee. I think the reason the argument comes down to GW=mostly good (despite slaves!) is because he is most known and respected for 1) fighting in the Revolutionary War for American independence, which modern Americans view as a righteous cause, and 2) being our first President. The argument comes down to Lee=mostly bad (plus slaves!) because he is most known for 1) fighting in the Civil War for the cause of keeping slaves, which most modern Americans view as a morally wrong cause.
The question of what they are most known for is an important one, because that is usually the same reason their statue was put up in the first place. When it comes to the question of whether to take one down, people tend to base their opinion on the questions on 1) what it meant when it was put up in the first place and 2) what it means now, in the context of history. With GW, it was put up because of his role in the Revolution and as President. With Lee, it was put up during an era of brutal reinforcement of white supremacy (see comments for link discussing this history) with a purpose of intimidating recently freed slaves. Today, in the context of history, GW’s statues are widely seen as a reflection of his leadership and role as a founder, not his role as a slave owner. Most people don’t go to a GW monument for the purpose of celebrating his slave ownership. Today, though, in the context of history, Lee’s statues are commonly given two negative meanings: First, they serve as a reminder of white supremacy to black people, and second, they serve as a rallying point for actual white supremacists. Yes, to many people, it may mean a “commemoration of Southern history” too, but if it’s 50% a brutal white supremacist reminder/rallying point and 50% Southern history commemoration, that’s enough to justify it being removed. We have made a moral decision as a society that its (even partial) role as a white supremacy beacon is not acceptable, in response to a particular flash point of a white supremacist resurgence. We have not made a similar decision about the Washington statues, because there has been no recent flash point around those.
However, I can’t actually morally justify Washington owning slaves, and that practice is indeed so reprehensible that it is valid to argue that if slavery is that wrong, then we should take down the statues of any slave holder, no matter how “good” they were otherwise. Joe Paterno’s statue was taken down because his biggest moral failing—protecting a child predator—outweighed the other good he had done. Perhaps the removal of Washington (slave owner) and Jefferson (slave owner and likely slave rapist) is the morally correct thing to do. We would likely remove the statues of contemporary heroes (say, MLK or Wayne Gretzky) if we suddenly found out they were rapists or owned slaves.
But there is a distinguishing factor between how we judge the actions of contemporaries compared to how we judge those of historical figures, and that is the factor of relative morality of a time in history compared to the present. Those who argue “slave owners weren’t all bad people” are inherently taking this factor into account. Yes, we all view slavery as evil now, but when it was a somewhat normalized aspect of society, it is plausible and even likely that many slave owners tried to live what they thought were upstanding moral lives in many ways. They may even have had moral dilemmas about slavery but felt that it was an intractable problem for them to solve, let alone forgo participation in. “Slave owners were not all bad people” ( a typically conservative argument) is a very similar argument to “George Washington’s statue should remain up because he did other good things, even though he owned slaves” (an argument liberals are currently making in relation to the confederate statue issue). “George Washington was not all bad,” essentially.
It seems that the right thing to do is to take down the Confederate statues because the of the bad things they were best known for (explicitly fighting for slavery), plus the reasons they were put up, plus the reasons they cause people pain now. But we must also admit that it would be logically consistent to remove other slave owners, even our founding fathers, if some contemporary flash point were to bring the issue of how bad slavery really is to the forefront. Perhaps it is a moral failing of our current time that we have not come to this realization yet. Perhaps future generations will come to the consensus that the founding fathers’ statues should be removed and hold it against our generations that we did not. Perhaps they will judge us harshly for tolerating other injustices, like unequal women’s rights and queer rights for so long. Societal morals evolve over time. In the near term, though, it is likely that the “contemporary, widely-held perception of the statues” factor and the “relative morality of the time of the person” factor saves the Washington and Jefferson statues now but not the Confederate statues. So down with the Confederate statues. And shame, at least, on the moral failings of those whose statues we leave in place.