(n.) a sandcastle
A dilly-house is a sandcastle—or, figuratively, a room filled with too many knickknacks.
Given that sandcastles are often bedecked in stones, shells, bits of seaglass and the like, it’s easy to see how that second meaning might have come along here. But how did the word dilly comes to mean ‘sandcastle’ in the first place? Well, that’s a trickier one.
The short answer is that we don’t know. But there are a few possible theories. Dilly is both an old nickname for the daffodil flower, and for a light carriage or coach (originally derived from a jocular play on the ‘diligence’ of the coach driver). By extension, it’s also an old word for a wheelbarrow or some similar contraption, used to transport agricultural equipment or produce. None of these seems to have any connection here, however.
Dilly is also an old childhood nickname for a duck. There could be a waterborne connection here, but that seems unlikely given that (with the possible exception of eider and sheldrakes) ducks aren’t readily too readily associated with sandy beaches.
A more likely theory is that the dilly here is the same as in dilly-dally, a word which has been used to mean ‘to waste time’ since the 1500s. Etymologically, it’s just a fanciful extension of the earlier word dally (derived from the French verb dalier, to chat), which has been used to mean ‘to amuse oneself’ since the Middle English period.
This dilly carries a sense of something playful, childish, trifling, or purely for amusement—and that, potentially at least, might make a dilly-house a house built just for fun.