Michael Signer said he was disappointed the white nationalists had descended on his town and said responsibility lay with Mr Trump for inflaming racial prejudice during his presidential campaign last year.
Mr Signer said: "I'm not going to make any bones about it. I place the blame for a lot of what you're seeing in America today right at the doorstep of the White House and the people around the President."
He was speaking shortly before it emerged one person had been killed and several others injured when a car hit anti-fascist protesters.
The clashes broke out at a "Unite the Right" protest over the decision by the local government to remove a statue of the leading Confederate general, Robert E Lee, during the American Civil War.
Mr Trump has now issued his first tweet condemning the violence, several hours after the violence began.
We ALL must be united & condemn all that hate stands for. There is no place for this kind of violence in America. Lets come together as one!— Donald J. Trump (@realDonaldTrump) August 12, 2017
He followed this up with another, calling the clashes which were centered around the city's university, "sad".
Am in Bedminster for meetings & press conference on V.A. & all that we have done, and are doing, to make it better-but Charlottesville sad!— Donald J. Trump (@realDonaldTrump) August 12, 2017
Some on Twitter accused the President of not being specific enough in his tweet as he did not specify that he was referring to white supremacists or far-right groups.
Several members of the Klu Klux Klan joined in with the protests as well as various alt-right activists, militia members and Confederate heritage groups.
Love how vague you kept this as if to say "both sides are bad!" Nazis are marching in your name. Be a man and take a real stance.— Alex Leo (@AlexMLeo) August 12, 2017
Mr Trump has repeatedly been accused of having stirred up racial tensions in the country through his rhetoric on immigrants and his vow to build a wall between the US and Mexico.
That's it? That's you lukewarm response to white supremacist at the back door of the White House wrecking havoc?— Bishop Talbert Swan (@TalbertSwan) August 12, 2017
Officials in Charlottesville declared a state of emergency shortly after 11am local time as the clashes became more violent, with punches thrown between rival groups and pepper spray released in the crowd.
The clashes are the latest since the city, situated around 100 miles from Washington DC, voted to remove the statue from a park.
So-called alt-right journalist Jason Kessler said the protest was partly about the removal of Confederate symbols but also involved wider issues around free speech and "advocating for white people".
He said in an interview: "This is about an anti-white climate within the Western world and the need for white people to have advocacy like other groups do."
More attendees are expected to flock to Charlottesville - normally a quiet, liberal-leaning university city.
Both local hospitals said they had taken precautions to prepare for an influx of patients and had extra staff on standby.
The Anti-Defamation League and the Southern Poverty Law Centre, which track extremist groups, both believe the event has the potential to be the largest of its kind in at least a decade.