'Decades later and still not ruined by tourists' – David Mitchell's enduring love affair with a Japanese island

My dog-eared, 1990s Lonely Planet guide to Japan described the Oki Islands as “remote and rarely visited”. Irresistible. I boarded a ferry one rainy dawn at a working harbour in Shimane Prefecture, on the Sea of Japan coast. The 50-mile crossing was choppy and the vessel shuddered as it slapped the troughs between the rolling waves.

There were no obvious tourists in the communal cabin area – shoes off – and the only foreigner was by now wondering if “remote and rarely visited” had been a coded warning. After a couple of hours the weather brightened and I went out on deck. Dogojima, the largest of the four inhabited Oki islands, prised the horizon open. Its landscape was, and is, ruckled with low mountains, cloaked in forests – and secretive.

Saito, the main port, slid into view around a headland. The sight stirred my 25-year-old imagination. Who lives here? What are their lives like? What are their stories?

Hundreds of ports dot Japan’s indented coastline, and many are a lot like Saito: sleepy, rusting, a decade or more behind the big cities along the Pacific coast. You can walk up and down the main street without realising it is the main street. You’ll find a squeaky clean convenience store; a tiny boutique for a silver-haired clientele run by a silver-haired shopkeeper; a noodle joint that gets busy over lunchtime, but which is closed by 4pm. Nobody is in a hurry.

Fishmongers sluice down the pavement in front of their shops. Vending machines display soft drinks discontinued elsewhere. A barber will do you a cut for 500 yen (£3.70) if you don’t mind cigarette ash in your hair. Cats yawn on sunny walls. Wildflowers sway in the courtyards of ramshackle shrines to Ebisu, god of good fortune and fishermen.

On hot afternoons, front doors are propped open with a wader, and you might smell home-cooking as you pass, or overhear someone practising pianist scales. Rarely do you feel more like an outsider, but if that bothered you, you wouldn’t be here.

The twice-a-day bus trundled along the coastal road anticlockwise, through villages whose faded signposts I couldn’t read anyway, past boats and submerged rice fields that shone like mirrors. Side roads wound inland and uphill. Maybe they led to farmyards, or just narrowed into tracks, then paths, then just fizzled out.

It was near Boys’ Day – May 5 – and carp streamers were flying from giant poles staked out in gardens. I’d never seen so many of these fish-shaped windsocks, swimming in the afternoon breeze. The island bus doubled up as a school bus, so I was met by amazed young faces and a few “Hello how are you, I’m fine thank you and you?”s from the bravest. Gales of giggles followed from the rest. I just sat there, failing to look enigmatic. When confronted by a posse of high-spirited sixth-graders, discretion is the better part of valour.

The bus driver pulled up at a stop, turned around and told me “Fuse!” to tell me I’d arrived. (“Fu” as in “fool” plus “se” as in “said”.) The village was a couple of rows of houses clustered around a beach. The biggest and oldest had ceramic-tiled roofs. Newer, smaller places looked flimsier and disposable. A small dusty shop sold everything. (When I read Akira Yoshimura’s unparalleled short novel about a fishing village in pre-Meiji Japan, I used the topography to “mind-stage” the book.)

The youth hostel, the only accommodation on the island I could comfortably afford, sat at the top of a slope. It looked like a former barracks. Nobody was around. I stowed my backpack and followed a path over a ridge and down to a rocky cove where I had my first ever swim in the glassy Sea of Japan. Afterwards, I tried to read, but the birdsong, dappled shade and the sound of waves pulled me under.

Later, back at the hostel, an old woman with a beatific smile had appeared like a character in a Miyazaki animation. I couldn’t begin to crack her dialect, but she didn’t let that stop her talking. She prepared a meal. Youth hostel meals are more miss than hit, but my journal confirms my memory that it was amazing.

Fresh scallops, prawns, seaweed salad and turban shells, thereafter my all-time favourite mollusc. The only other guests at the hostel were a couple from Yokohama and their very young daughter who called me “Nii-chan”, or big brother – the first time I’d been awarded the moniker. We swapped addresses but, pre-internet, scraps of paper got lost. I still wonder what hand life dealt them, and hope it was a kind one.

The following morning I rented a bicycle two or three sizes too small for me. I must have looked like a freshman cycling off to clown school. My memories of the next few hours are an impressionistic mash-up, but so are my notes and this was 25 years ago. I remember a hill park, half-misty, half-sunlit, where azaleas bloomed magenta, mauve, flamingo pink, orange and white. Lurid spiders you wouldn’t want to mess with waited in webs woven across the overgrown paths.

I remember hiking up a trail for a view of the Tokage Rock – an outcrop resembling a giant lizard clinging to a cliff. Kind of. (It’s the taking part that counts.) There was a polyphonic waterfall that hypnotised you, if you let it. There were several tunnels which I traversed along the raised access walkways. Water dripped from the roof. One was a few hundred metres long and curved, so I couldn’t see the end. When a vehicle entered the tunnel, its walls trapped, echoed and amplified the engine’s sound waves to astonishing decibel levels. I couldn’t let myself be as afraid as I felt, and when I emerged, buzzing with adrenalin, I felt freshly returned from the underworld.

Lastly, I remember a mountain shrine. It was in the midst of giant trees thicker than columns in a cathedral. I sat there in the company of the place, its depths and silences, and my mind, and the flora, fauna and the Shinto deities, if you believe in them, or half-believe. An hour passed.

Half my lifetime passed. I went back to Dogoshima 18 months ago. Saito is a little less sleepy and there’s a new ferry terminal connected to a hotel and visitor centre via an overhead walkway. Unlike Kyoto, Miyajima or anywhere on the TripAdvisor Top 20, the Oki Islands are still themselves, and still many thousands of travel articles away from being spoilt. I rented a car and stayed in a hotel, but I couldn’t find the mountain shrine. Perhaps, like the youth hostel at Fuse and its elderly warden, the shrine is simply gone, and can only be accessed via a quarter of a century ago. Occasionally I dream of it. Perhaps I left a fragment of myself there.

David Mitchell’s new novel Utopia Avenue is out now (Sceptre, £20). To purchase a copy from the Telegraph visit books.telegraph.co.uk or call 0844 871 1514.