David Mitchell and I are talking – nerdily, greedily – about a moment in popular music when prog rock, folk rock, acid jazz and psychedelia all bubbled jauntily to the surface of the cultural pot. I’m telling him about how the guitarist and songwriter John Martyn ended his days in the small town near where I live in rural Kilkenny; he’s filling me in on how Jimi Hendrix’s bass player, Noel Redding, lived out his life in Clonakilty, the seaside town in County Cork where Mitchell lives with his wife Keiko and their two children. Redding, he says, continues to dominate conversation: “In people’s memories and anecdotes, he’s still walking the streets now.” But neither of us was there for the heyday we’re remembering, and which provides the setting for Mitchell’s eighth novel, Utopia Avenue, its title the name of his invented four-piece band. In the late 1960s, both of us were just being born. So why the fascination with recreating this precise period? Well, says Mitchell, the music was great. Sure, and? “Things were happening for the first time. The album as an art form, rather than just as a convenient way to store singles and lesser tracks, kind of gets invented by Sgt Pepper’s. Nothing comes from nowhere, just like the novel, but it’s a strong contender for the first of its kind. This idea that you curate songs and make a musical journey out of them – imagine being in a year when that gets invented, and now there are lots of them, popping up like mushrooms when the perfect cultural combination is right, when the conditions are right.” Writing Utopia Avenue, Mitchell became especially interested in Brian Eno’s concept of “scenius” – the idea that a grouping of artists in a particular place at a particular time “can have a genius, where marketing arrangements, business models and of course a pool of hungry young talent in tune with the zeitgeist, just the few times in history, it all comes together perfectly. It never lasts long, it’s soon commoditised and exploited and imitated and it dissipates again until the next time it pops up elsewhere.” This makes a lot of sense, because Mitchell’s novels, from his 1999 debut Ghostwritten, through Cloud Atlas and Black Swan Green in the mid 2000s and 2014’s semi-supernatural epic The Bone Clocks, are strongly preoccupied with the idea of convergence; that moment when apparently random coincidence is revealed to be something more like a heady brew of cosmic echo, historical recurrence, personal destiny and narrative fillip. Stories nested within stories are a hallmark of his work, most notably in Cloud Atlas, which he laughingly calls “my biggest album, if you like. Some more recent albums have done pretty well too, but it’s going to be there on my tombstone and it paid for the house, and I’m very grateful. And I know it: I’m the Cloud Atlas David Mitchell as opposed to the comedian David Mitchell.” Characters recur across novels; readers of The Thousand Autumns of Jacob de Zoet and The Bone Clocks will immediately sniff the wind when Dr Marinus makes an appearance in Utopia Avenue – not merely because they recognise him, but because he alerts us to another dimension in what is, elsewhere, a largely realistic novel; the otherworldly battle between the centuries-old representatives of good and evil. This is where Mitchell’s work starts to show its kinship with the concept albums of progressive rock: the grandiose and operatic sit down with the everyday and mundane; narrative swagger can be cheeky, campy, almost kitsch. Nowhere is this more in evidence in Utopia Avenue than in its procession of cameos by real-life musicians: David Bowie, Leonard Cohen, Frank Zappa and Brian Jones all flit across its pages, Bowie bumming a cigarette because “Lennon cadged my last one. Like the Scouse millionaire he is”. When I first started reading it, I thought I’d successfully identified Utopia Avenue’s lead singer, the poetic Elf, as Fairport Convention’s Sandy Denny, only to have Denny turn up a few pages later (Mitchell does soothe my readerly vanity by conceding that Utopia Avenue do have similarities with Fairport Convention, and with Fleetwood Mac, especially in the way they brought together different songwriting styles). The cameos, he says, were great fun – but he was careful not to overdo them: “I want enough of them in there just for the joy. I want them to be a little bit more than a face, I want them to say something that might slightly alter the direction of a scene, but I don’t want to write whole new chapters in the real lives of these real people, because their existences are not my raw material.” There are moments, naturally, when the passage of time has dramatically changed our understanding of someone; it’s not possible, for example, to read scenes featuring Jimmy Savile as a host of Top of the Pops, without 21st-century knowledge. Mitchell approached this portrayal by studying YouTube clips, and by practising restraint: “The important thing with writing him is to resist a temptation of putting in a chill, a dark shadow crossing the room that I’m imposing on to the scene. It all has to come from him.” Similarly, the business of writing about music – akin, as he admits, to dancing about architecture – had to be achieved with a light hand, as in two novels he admires, Salman Rushdie’s The Ground Beneath Her Feet and Iain Banks’s Espedair Street. “A page of description about a non-existent song, ways of describing how the instruments sound, that would just kill a chapter, I think. A page would, but three sentences, you’d maybe get away with. So I tried to stud the more musical scenes with that, with a very short number of sentences describing music, just to give a ghost of the sound for the reader, to give them a textural impression of a song.” But Mitchell wanted Utopia Avenue to be more than an intricately observed portrait of the Soho music scene of the late 60s and early 70s. He was also, he tells me, interested in exploring where an artist’s ideas come from, and how they are brought into the world; and pop music offered him a direct and vivid opportunity. “I think most writers have a deep-seated envy of most musicians, because they perform and they’re a part of the appreciation feedback loop,” he explains. “As they perform, they interface directly without any time delay whatsoever with the consumers of their art … I can see how glorious it must feel – we all can, just when we look at the high of performers on stage. And however brilliant a scene I write, however perfect a paragraph I compose, I will never ever, ever, get a taste of that drug. The closest is just a fairly weak imitation if you read something on stage that goes down really well, and there’s an audience there, that’s nice.” In fact, I’ve seen Mitchell surprise audiences at literary festivals with a story he’s never read aloud before – and it seems to be the precise opposite of that sinking feeling you get when a band announces they’re going to play some new material when you’re yearning for the hits. But he’s right: “It’s not 15,000 people just screaming ‘yeah, yeah, yeah’ at you at the top of their voice.” On the other hand, he values the freedom that comes with being a writer, even that rare thing, a literary bestseller. He feels, he says, under “no commercial pressure at all, in fact, if anything it’s the reverse, which as it happens is great, because it harmonises with my own wishes anyway, to not write that greatest hit again and again and again. For musicians that’s hard. They need a very, very large fanbase to want them to move on and not sound pretty much like they first did when that fan first fell in love with them.” It’s hard to imagine Mitchell in the role of the bumptious rock god, pouring out the hits for the slowly ageing groupies; he’s the image of self-effacing, good-humoured gentleness. He frequently corrects himself, picks up on his poor grammar or momentary vagueness, and is exceptionally tactful when one of my questions collapses under the weight of its own abstractions (“We might need to co-operate to give it more form”). It seems likely, then, that his creativity would always take a somewhat more reflective, inward form. When he was growing up in suburban Worcestershire, reading Tolkien and Isaac Asimov, Susan Cooper and Ursula K Le Guin, his mother, a commercial artist, would lay out pieces of A1-sized paper to keep him occupied during the school holidays. Now, he sees the imaginary maps that he drew as “proto-novels”, his impetus “to make something enormous, to make something as gut-grippingly vast as Middle-earth, or Earthsea”. (His mum, incidentally, continues to have an impact on his creative output; Utopia Avenue, both band and book, were going to be called The Way Out until she told him that sounded like an advisory service for people considering suicide.) What he likes as a grown-up is to hop between this kind of maximalism and the more contained units of individual novels – or, as he puts it, to have his cake and eat it. As a child, he struggled to overcome a stammer, an experience he has revisited in fiction; as a young man, he taught English in Japan, launching himself into a foreign-language environment and at one point taking six months out of work to travel the Trans-Siberian railway. There is, maybe, something in his personality that relishes shifts and contrasts. While he was teaching in Hiroshima, he met his wife, Keiko Yoshida, and they continued to live in Japan before moving to London and then Ireland. The couple have a daughter and a son, who has autism. Mitchell and Yoshida worked together to translate The Reason I Jump by Naoki Higashina, a Japanese boy with autism, in 2013. Every novel, he maintains, must stand alone, and be accessible to people who have never read a word of his before, and might never again; in that process, “I get to research and world-build and cast-assemble every single time, afresh, and that’s great, and I love doing that.” But each time, he’s engaged on a parallel project: “I also get to work on my own private Middle-earth, built of the books, a kind of an uber-novel as it’s been called, in which each individual novel I publish is a new chapter. I don’t know always where it’s going, or even why, but it brings me pleasure to see it getting larger and more inked-in and thought-out and a bit heftier. And it’s of course building up to some sort of war in the future, but I feel like I’m assembling the pieces of some kind of chess board. It’s not chess, and I don’t know what the rules are and I don’t even clearly know who the antagonists and the protagonists will be, what the sides are or what’s at stake.” There is something touchingly boyish about this sense of world-building, as if Mitchell is still creating on sheets of A1 – even though when he talks about writing and what he calls, perhaps half-jokily “the republic of letters”, he’s canny and perceptive, noticeably un-waffly. Mitchell says he doesn’t “really want to pour my creative years into a single Middle-earth or Westeros”, and that while he’s grateful other people do, he’s too attracted to jumping between projects. But his fidelity to the idea that there is a connection between all his endeavours is striking. When The Bone Clocks was published, he told the Guardian that he had his next five books planned out. In the quiet of Cork, the uber-novel slowly takes shape. • David Mitchell will discuss Utopia Avenue at a Guardian Live online event and Q&A on 14 July.