Damme’s stately 15th-century City Hall, made of limestone while everything else is made of brick, points to the wealth that herring fishing once brought into the town.