State of the nomination: Democrats get serious as 2020 field narrows?

For the historically large and diverse field of Democrats seeking the presidential nomination, it has been a long hot summer. But with the arrival of the Labor Day holiday comes a time-honored transition to a new and more volatile stage of the race. The field is narrowing, the competition growing more sharp.

Related: How Warren surged past Sanders – and how he fought back

Former vice-president Joe Biden has held his early lead despite a season of missteps and gaffes. He is trailed by Vermont senator Bernie Sanders, who has staked out healthcare as his defining issue. But both have lost ground to Massachusetts senator Elizabeth Warren, who spent her summer churning out policy plans, working selfie lines and steadily climbing the polls.

Rounding out the top tier are the California senator Kamala Harris and South Bend’s mayor, Pete Buttigieg, who used televised events to break through a congested field but have seen their progress stall.

Party leaders and strategists say the race remains fluid at the top as the contenders prepare to share a debate stage in Houston in two weeks’ time.

“The stakes are higher now but the race is still open,” said Tom Vilsack, secretary of agriculture under Barack Obama and former governor of Iowa. “A spark or a moment during the debate could catapult one of them forward.”

Polling last week showed Biden held a comfortable lead at 32%, with Warren and Sanders jostling for second. A USA Today/Suffolk University survey found Warren at 14% and Sanders at 12%; Quinnipiac showed Warren at 19%, four points ahead of Sanders. In both polls, Harris and Buttigieg were the other candidates to top 5%.

But as Barack Obama and Donald Trump showed in 2008 and 2016, early assumptions and public polling are not always good indicators of who voters will choose.

Biden has an advantage among black voters but faces competition from Harris and the New Jersey senator Cory Booker, the most prominent African American contenders who have targeted his record on race.

“The nightmare scenario for Joe Biden is that Kamala Harris or Cory Booker emerge very strong in the Iowa caucuses and that leads to an erosion of support among African Americans,” said Bill Galston, a senior fellow at the Brookings Institution.

The biggest moment of Harris’ campaign so far was the first debate, when she clashed with Biden. But she has faded since. Galston said: “I think Kamala Harris’s supporters are wondering: does she have a second act after the first act was such a success?”

Biden has portrayed himself as the man to beat the president. His first TV ad hammers the point: “We have to beat Donald Trump and all the polls agree, Joe Biden is the strongest Democrat to do the job.” But many argue that electability, a calculus that can include gender and racial bias, is not enough on its own.

“We can’t nominate someone who we’re not excited about just because we’re afraid and then turn around and ask the rest of the country to get excited about them,” said Heather McGhee, a fellow at the progressive thinktank Demos.

Democrats are looking for “an exciting alternative to not just the last four years but also the last 40 years of rising inequality and insecurity”, McGhee said. “We need someone who can meet the populist moment.”

The field crosses the ideological spectrum, from democratic socialist to conservative Democrat; spans four decades in age; includes several women, candidates of color and a gay mayor; and contains veterans and a diversity of faiths and socio-economic backgrounds. That, said María Urbina, national political director for Indivisible, a progressive grassroots organization, has stoked a “real contest of ideas”.

“We’re not having just a ‘binary of good and bad’ debate,” she said. “It really allows for a serious discussion of the issues. Candidates actually have the space to talk to people, hear their stories and offer solutions.”

One of the central debates is over how to expand healthcare coverage. Differences between candidates who advocate for a single-payer system and those who want to expand the Affordable Care Act have led to sharp exchanges.

Warren has made economic reform central to her campaign. Sanders champions ideas such as Medicare for All and the Green New Deal. Others have forced conversations on issues such as reparations for slavery, court packing and decriminalizing border crossings.

Michael Steele, a former Republican National Committee chair, believes Democrats must not veer too far left. He argues that moderate and independent voters in battleground states want a candidate who will re-establish order and stability after four years of Trump. A nominee who wanted to overhaul nearly every aspect of the US economy would be a “second shock to the system”, he said.

“The question is whether or not that’s something that’s palatable to the American public,” he said. “My bet is it is not.”

‘Three Stooges’

The field has been reduced to 20. The New York senator Kirsten Gillibrand, once seen as a major contender, ended her campaign after failing to qualify for the third debate. Washington governor Jay Inslee, former Colorado governor John Hickenlooper and the Massachusetts congressman Seth Moulton also called it quits.

Half of the remaining candidates will not debate in Houston, raising serious questions about whether they can win the nomination.

Trump continues to express confidence, despite warnings of a recession and new Republican challengers. Former Massachusetts governor Bill Weld has been joined by former Illinois congressman Joe Walsh, a disillusioned Tea Party star. There are rumblings that former South Carolina governor and congressman Mark Sanford will jump in too.

“Can you believe it?” Trump tweeted. “I’m at 94% approval in the Republican party, and have Three Stooges running against me.”

Trump is also watching the Democrats. Tongue firmly in cheek, he said Gillibrand’s exit marked a “sad day”, adding: “I’m glad they never found out that she was the one I was really afraid of!”

Voters feel less playful. A USA Today/Suffolk University poll that asked respondents to choose the word that best captured their sentiment ahead of the looming campaign saw just 2% of Democrats saying they were “optimistic”. Only 1% felt “confident”; 16% said they were “worried, scared, frightened or afraid”.

Neera Tanden, president of the Center for American Progress, a left-leaning thinktank, said the trauma of 2016 had left Democrats with little appetite for a long primary fight.

“The Democratic electorate is so anxious and concerned about the election with Trump, that they will start voting strategically to end it in March or April,” she said. “They don’t want this to go one forever.”

If Labor Day is the start of a sprint to Iowa and New Hampshire, candidates must not be caught flat-footed. On Monday, Biden will attend picnics in Iowa while Sanders will zip from breakfast in southern Maine to New Hampshire, where he will march in a holiday parade before cooling off at ice cream socials.

Warren will make her 15th visit to New Hampshire, for a house party. Harris will join union members and healthcare advocates at a rally in her home state, California.

• David Smith contributed to this report