Sim and Madley Croft would record their vocals straight on to their laptops at home, trying not to wake their parents; Smith mixed the songs in a tiny room under the XL offices in Notting Hill.Its dark, moody sound deepened as a result of this nocturnal process.
Their album recently went platinum (325,000 copies have been sold to date) and its songs have been covered by stars as various as Damon Albarn, Shakira and Orchestral Manoeuvres in the Dark.
That they have achieved this success while keeping themselves to themselves is quite a feat in our flaunt-yourself pop culture.
Thick snow clings to the banks of the Thames and icicles hang from wrought-iron railings.
The xx used to call this their manor, but they have been away touring the world for the past 12 months - a year whose peak came in September, when their debut album, xx, beat Paul Weller, Dizzee Rascal and others to win the Barclaycard Mercury Prize.
As Madley Croft reminds me, they made an album full of personal sentiments that they initially thought only "four people would hear". This was pre-fame, when they used to rehearse in a tiny space under the railway arches, where sticky carpet covered the walls and empty Coke cans littered the floor.
Baria Qureshi, then the fourth member of the band - whose departure last autumn Madley Croft still describes as feeling "like a divorce" - was there, too.
The band's melancholic songs, with lyrics about loneliness, lust and love, have struck a chord in Britain in particular, providing the perfect soundtrack to our troubled times.
The BBC chose their song "Intro" for its election coverage - and their popularity was even latched on to by the Tories, who used their music without permission at the Conservative party conference in October.
Instead, they have worked their way into our collective consciousness by inventing an urban sound that unites the emotions of indie rock with the drama of dance music. ' We made this record for ourselves - to try to make music that all of us would like." Perhaps their success is due to the way their combined influences tick a range of critical boxes - or, less cynically, maybe the mood of intimacy and familiarity they inspire is something that people crave, especially in a world bombarded with 24-hour content.