Jenner, 39, shared a gallery of throwbacks in tribute to Stoker on Instagram Tuesday, tracking her pregnancy as well as rare photos of their fraternal twin sons, Bo Thompson and Sam Stoker, whom they welcomed in February 2020. Jenner is also dad to 5½-year-old daughter Eva James, whom he shares with ex-wife Leah.
"We just knew it right away, didn't we? That we would spend our lives together and raise a family. Be best friends and adventure through life together. Happy birthday, my love," he writes in the caption. "Thank you for the incredible moments we have shared thus far, and for the love, laughter and learning that has yet to unfold."
"Eva, Bo, Sam and I are the luckiest people in the world ❤️ 🎂 @caylita_ 🦋," he adds.
Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
In the comment section, Jenner's mother Linda Thompson wrote: "Happy birthday to your Spectacular wife @caylita_ ! The mother of your two beautiful sons and a gorgeous goddess of life! Thank you Brandon for bringing such an incredible woman into our lives… I love Cayley's whole family and I'm very grateful… Cayley you rock! I love you!"
The couple announced they were expecting in August 2019. Brandon told PEOPLE at the time, "We're madly in love and we are very excited about this!"
He also recalled what it was like telling his daughter she was going to have siblings.
"We told her together," Brandon said. "Leah, Cayley and I sat down with her and told her what was happening, and she was very excited and happy. She's super excited to be a big sister."
Brandon, the son of Caitlyn Jenner and Thompson, and Leah, whose father is former Eagles guitarist Don Felder, wed in May 2012. Brandon and Leah announced their split in September 2018, after 14 years together. Since then, Brandon said the two maintain a great friendship and co-parenting relationship.
"Leah and I split up in a really beautiful way, and I think we handled it in the best way possible," he previously told PEOPLE. "It was a process, and it took some time to finalize. And now that Leah and I have moved on, I am starting a family with Cayley, and I'm so happy about it."