It's tricky business judging historical figures outside of the context of their times, but even then the evils of slavery were well known and many people were appalled by it. That's not to say that many who opposed to it were not racists at least by today's standards. Those who could reach out and overcome their conditioned biases were much more rare than they are now and most abolitionists were driven by religion. Once Lincoln actually met a black man like Fredrick Douglass his personal attitude started to rapidly evolve, he didn't shut down, but was open and empathetic. This wealthy railroad lawyer became involved with the republican party because of abolition and ended up with the 14th amendment. Was he perfect, no not by a long shot, but he was a far better man than most. Stripping away the myth, he did more good than bad and was a driving force for liberalism.
Most union soldiers were racists, the vast majority of white ones were not too different than the southerners they fought against. As president, Lincoln's first priority was the preservation of the country, the union and everything else was of secondary importance. Once the north got on it's feet militarily, was geared up for war and the threat of dissolution diminished, the priorities changed, to the point where the 14th amendment caused near panic in the south. Freeing the slaves was one thing, making African Americans equals with the franchise was something they never dreamed of in their worse nightmares. Native Americans never had the franchise either, but that changed too.