Microsoft Build recap: Team Copilot, a Sam Altman surprise, and other AI announcements from the developer conference

• Microsoft's Build 2024 keynote for developers happened on Tuesday.

• CEO Satya Nadella gave updates on Microsoft's generative AI products, like Copilot.

• Major announcements included Team Copilot, GitHub Copilot extensions, a Khan Academy partnership, and a surprise Sam Altman interview.

OpenAI and Google may have held the AI spotlight last week, but now it's Microsoft's turn.

The annual Microsoft Build developer conference keynote lasted over two hours on Tuesday, during which CEO Satya Nadella unveiled Microsoft's latest generative AI offerings, including Copilot updates.

Copilot was the dominant theme, continuing what Microsoft started on Monday when it held a press event and unveiled its new Copilot+ PC lineup, which includes the AI-infused Surface Pro and Surface Laptop. Focusing on hardware announcements on Monday gave Microsoft plenty of airtime to focus on AI software and features during Tuesday's keynote.

Microsoft unveiled Team Copilot, bringing its AI agent to workplace chats and meetings within Microsoft Teams. Team Copilot is designed to be a "valuable member of the team" that helps people be more productive at work, according to Microsoft.

Other major announcements included GitHub Copilot extensions, a Khan Academy partnership to bring free AI tools to teachers in the US, and real-time intelligence capabilities coming to Microsoft's AI analytics platform, Fabric. Microsoft also said it was "deepening" its partnership with Meta to bring volumetric Windows apps to the Quest headset, extending the apps into 3D space.

But the biggest surprise came at the end of the keynote when OpenAI CEO Sam Altman came to the stage to talk about GPT-4o and his advice to startup founders and developers. Microsoft has invested billions into OpenAI as its partner, and that means GPT-4o is directly integrated into Copilot and the Azure AI cloud suite.

Business Insider liveblogged the entire Build 2024 keynote so you can catch up on the highlights below.

And that's a wrap on the Build 2024 keynote!

Microsoft's CTO Kevin Scott thanks Sam Altman for joining, and then closes things out by thanking the Build developer crowd one last time.

And that's it for the main keynote, which clocked in at over 2 hours.

Build 2024 continues with developer sessions in the coming days.

But that's a wrap on our live blog — thanks for joining!

A surprise guest: Sam Altman!

We're getting a surprise interview with OpenAI CEO Sam Altman to close out the Build 2024 keynote.

Sam Altman takes the stage and says it's been a "wild week" since OpenAI showed off GPT-4o, its new flagship AI model.

"I've never seen a technology get adopted so quickly in such a meaningful way," Altman says.

Altman says he was genuinely surprised how much he liked ChatGPT's new voice mode powered by GPT-4o.

Altman says it's a huge deal that "the models are just going to get smarter."

The OpenAI CEO says his biggest piece of advice is that this is a special time, so take advantage of it and don't wait to build what you're trying to make. Now is the "most exciting time" to work on a startup since the mobile boom in 2008, and maybe even the internet, Altman says.

But don't expect AI to do all the work for you, Altman adds.

"AI alone is a new enabler but it does not automatically break the rules of business," he says.

AI has huge potential to reduce suffering in the world, Microsoft's CTO says.

Kevin Scott asks the developer crowd to think big with AI, and especially about the potential for AI tools to "reduce suffering" around the world.

For example, he wonders if AI could help his mother with her sickness. It would be great if she could upload her medical charts for AI analysis and get some smart test recommendations, Scott says.

Khan Academy's CEO comes out to chat about its new partnership with Microsoft.

"This is a big deal," Khan Academy CEO Sal Khan tells the Microsoft CTO.

Every teacher in the US will have access to Khan Academy's "state of the art" teaching tools for free, Khan says.

He says he thinks teaching will be the first profession to really benefit from AI, and if they can win them over, students will follow.

CTO Kevin Scott shows off a GPT-4o demo that was recorded in the last 24 hours.

We see a Microsoft employee get some quick coding help with OpenAI's GPT-4o by using her phone camera to show the AI agent the Python code on her laptop screen.

The Scarlett Johansson-like "Sky" voice is gone, with a deeper, more robotic-sounding voice in its place. OpenAI pulled the "Sky" voice recently, and Johansson has since said that she was approached to voice GPT-4 last year but declined. After hearing a voice "eerily similar" to hers, she said she's lawyered up.

Scott says the OpenAI GPT-4o demo was conceived of and recorded in the last 24 hours, and its coding abilities give him hope that a "rusty" CTO could still dive back into coding if called upon to do so.

Microsoft CTO Kevin Scott is up next as we near the end of the Build 2024 keynote.

He thanks developers for "all of the great shit you have made over the past year."

He shouts out the over 50,000 customers using Azure AI tools and the platform, ranging from small startups to Fortune 100 companies.

Specifically, he highlights some generative AI work done with Etsy to build features like the marketplace's new "gift mode."

"It is a really innovative way to figure out how to buy things for people who are really difficult to buy for," Scott says.

He then zooms out and reflects on the huge technological shift that AI represents, comparing it to the dawn of the internet. He also shouts out OpenAI's new GPT-4o model.

If you just look at AI compute over time, Scott says, the rate of increase in compute when applied to AI training has been increasing exponentially — and the point of diminishing returns hasn't been hit yet.

Microsoft is working to optimize the current frontier while also investing at "a pretty incredible rate" to "push the frontier forward," Scott says.

Microsoft is "deepening" its partnership with Meta.

Pavan Davuluri announces that Microsoft is "deepening" its partnership with Meta to bring Windows apps to the Quest that extend into 3D space.

We're seeing a clip about bringing volumetric Windows apps to Meta's Quest headsets.

Heads up, gamers: Copilot can help you when you're stuck on that tricky quest.

Pavan Davuluri runs a clip showing how Copilot can work in Xbox games.

We see footage of someone asking Copilot questions about how to complete an in-game task, and the AI agent helps out.

And don't forget the "Minecraft" example we saw earlier.

It sounds like Xbox players won't have to Google as much if they get stuck in a game.

Next up: Copilot+ PCs, which were first unveiled yesterday.

Pavan Davuluri, Microsoft's Corporate Vice President of Windows + Devices, gets onstage to talk more about the big announcement from Monday: Copilot+ PCs.

Davuluri says the Copilot+ PCs are "redefining what you can do on a PC and setting the direction for the next decade of Windows."

He runs a sizzle reel about the new AI-powered computers, which integrate AI chips and processors along with AI apps, like "Recall," which acts as a "photographic memory" of sorts for your computer, making it easier to quickly find information anywhere on your PC.

"This new class of PCs is up to 20 times as powerful and 100 times as efficient for running AI workloads compared to traditional PCs from just a few years ago," he adds.

Next up, Microsoft's president of collaborative apps and platforms takes the stage.

Jeff Teper is here to talk about Copilot extensions and do some demos to show what it's like to create and use one.

This is the part of the Build keynote where the more in-depth technical demos come out, so our updates might be less frequent.

Teper talks about how you can "drill deeper" into a conversation with Copilot extensions, such as asking about a key feature of a delivery drone. Copilot will provide the necessary information quickly to cut down on having to ask a real person for that information.

Teper's next demo is in Sharepoint, as he creates a custom and secure Copilot extension.

You can publish Copilot extensions back to Sharepoint and Microsoft Teams afterward, where they're visible in a marketplace.

We're hearing a lot more about how Team Copilot works.

Team Copilot goes beyond the individual user to help with project management.

It's a "valuable member of the team," and gets added to your chats and meetings — such as a "meeting facilitator" to keep conversations productive. It can tell the team in a meeting when it's time to move on to the next topic, and takes notes that anyone can add to.

Team Copilot keeps up-to-date information, including timely items or open issues that need to be addressed.

"The entire team will be more productive and collaborative," the video about Team Copilot says.

These features will be available to customers in preview later this year.

We're now hearing from Lumen, a company using Copilot for Microsoft 365 to help their sales staff. What normally takes hours can now take minutes, Lumen says. They've seen an increase in outbound calls by 40%.

Nadella heads off, and Microsoft exec Rajesh Jha takes the stage.

Rajesh Jha, executive vice president of Microsoft's Experiences + Devices Group, takes the stage to talk more about Copilot.

He says that "nearly 60%" of Fortune 500 companies now use Copilot.

Microsoft has added more than 150 Copilot capabilities since the start of this year, he adds.

Nadella unveils Team Copilot and updates to Copilot Studio.

Microsoft's Team Copilot works in Microsoft's Teams workspace.

It can be your meeting facilitator when you're in Teams, creating agendas, tracking time, taking notes for you, Nadella says — it can even be your project manager.

He also talks about updates coming to Copilot Studio, where you can design your own Copilot AI agent and test it before you deploy it. You can then use the AI agent across multiple channels.

Next up: Microsoft announces GitHub Copilot extensions.

Nadella announces GitHub Copilot extensions.

We're seeing a live demo where GitHub Copilot is asked in Spanish to build Java code.

GitHub Copiot extensions are good at "keeping you in your flow across your entire day."

The GitHub Copilot Azure extension makes it possible to diagnose issues as you code and plan to implement changes.

We're also seeing the AI tool help come up with a development plan, and then getting its help in implementing the code for a developer.

Coders have long dreamed of "staying in the flow," Nadella says.

Microsoft announces Real-Time Intelligence in Microsoft Fabric.

Microsoft Fabric is the company's AI data analytics platform.

More than 11,000 organizations use it, Nadella says.

With the newly announced real-time intelligence capabilities coming to Fabric, customers can enjoy "instant actionable insights on streaming data."

Microsoft is partnering with Khan Academy.

Khan Academy will use Microsoft's Phi-3, a family of small language models (SLMs), to make math tutoring more accessible; Khan Academy is making its AI assistant Khanmigo free to all US teachers.

In a clip, Khan Academy CEO Sal Khan says he expects the partnership to have a "dramatic impact on US education."

Nadella on stage says he's very excited to see the impact of the Khan Academy partnership.

We're seeing another demo, this one about hiking shoes.

In the video, a man holds up a shoe to his computer and asks the Copilot AI agent if they're fit for a camping/hiking trip.

Copilot responds, and then the man tells the AI agent to identify the best shoe for the trip and add it to his cart. The man speaks Spanish for the next question and the AI agent responds in Spanish.

More than 50,000 organizations use Azure AI today, Nadella says.

"With Azure AI, we offer the broadest selection of frontier and open source models including LLMs and SLMs," Nadella says.

He mentions how GPT-4o is integrated into Azure AI already.

What OpenAI demoed last week, "that was pretty magical stuff," Nadella says.

OpenAI is Microsoft's "most strategic and most important partnership," Nadella says.

Copilot will also leverage GPT-4o, Nadella adds.

You can share your screen with Copilot, and have it help with whatever you're working on.

We're seeing a demo of someone playing "Minecraft" and talking to Copilot, which can see what's on the screen, to help the gamer craft and survive. It offers timely suggestions, like seeing shelter "now."

After the demo clip ends, Nadella announces that GPT-4o is now "generally available" on Azure AI.

Next up, AI infrastructure.

Microsoft has "the most comprehensive world infrastructure" for AI, Nadella says.

"We're on track to have our data centers powered 100% by renewable energy by 2025," Nadella says.

The number of countries where its cloud-computing Azure AI services are available has quadrupled, he adds.

Microsoft will be among the first cloud providers to offer Nvidia's latest Blackwell AI chips, he said. The company has a "deep partnership with Nvidia," Nadella says.

The first major topic is Copilot, and the Copilot+ PCs announced yesterday.

Nadella talks up the Copilot Library, with ready to use local APIs.

There are 40+ on-device models to use "out of the box," he adds.

He also shows a sizzle reel about Copilot+ PCs and the AI apps that are tightly integrated in the computers.

Nadella starts a round of applause for developers helping build out AI applications.

Nadella mentions some developer interactions over the past year that stuck out to him, and how the AI apps people are making are making an impact.

Nadella thanks those who "bringing the impact" of AI to the rest of the world.

Microsoft CEO Satya Nadella has taken the stage.

He's talking about key moments in Microsoft's history.

"It feels like we're yet again, at a moment like that," he says.

AI is reshaping "every layer of the tech stack," Nadella adds.

Microsoft has two real dreams, Nadella says: Can computers understand us instead of us having to understand computers? And, in a world of ever-increasing information, can computers help us reason, plan, and act more effectively on that information?

He says there are "real breakthroughs" on both fronts.

Ok here we go! The Microsoft Build keynote is kicking off.

We're seeing a video about the "new era" of technology to kick things off. There's a snapshot of OpenAI's text-to-video image generator, Sora, in the video.

Microsoft CEO Satya Nadella confirms lots of Copilot news is on the way...

"Looking forward to #MSBuild today, where we'll share how we're creating new opportunity for developers across every layer of the Copilot stack. Please join us," the Microsoft CEO posted to X, formerly Twitter, on Tuesday morning ahead of the keynote.

You can also watch a livestream of the Microsoft Build keynote.

While plenty of developers and press will be in attendance, Microsoft has also made a livestream of the opening keynote available for those who prefer to watch and listen in real time.

But we'll be covering the highlights here too.

Read the original article on Business Insider