I think this would prove too easy to use to its full potential. In my building deck, I often end up with a large number of recruits, and while gems are used, the bricks value does drop below the gem total rather regularly. It would be especially easy to play just after playing another +tower card.
As a Common, I think it should have an additional limitation. Perhaps not building beyond a certain height, or reducing the greater of the other two resources to match the bricks value.