I'm still looking at it from the perspective of going for the Resource Accumulation Victory. If someone with 54 tower can play this to gain more than 100 bricks, he's likely to win that way. Every Building deck would take this card. Even without it, my Building deck often wins with a Resource Victory. 118181 is a good example of that.
Making the brick return 1xN or less would prevent someone from playing cheap +tower commons and getting several times the return. I shudder at the idea of playing Lookout towers for 7 bricks, playing this one to gain 45 bricks, and then playing Lookout towers again. Foundations, Guard tower, Reuse the rubble, Tower, even Work in progress could be tossed in just to gain several times their cost in bricks, and these are just the Commons. Those bricks would then either fund an Uncommon card to rebuild the lost tower (near-guaranteed win with Pyramids or Wishing lagoon on hand) or be put toward a Resource Victory. It's too potent.