The FSB said that the number of banks had fallen by a fifth since the government-commissioned Cruickshank report, published in 2000, called the UK banking sector "monopolistic".
The business organisation spoke out ahead of the publication of the final report from the Independent Commission on Banking, which was set up by the government to find ways to stop a repeat of the banking crisis and is set to make its final recommendations to George Osborne in September.
The Cruickshank report found that four main high street banks - Lloyds TSB, Barclays, NatWest and HSBC - had an 83% share of the small business market in 2000; however, the FSB said that their share has since grown to 85% due to a steep decline in the number of UK banking groups and their subsidiaries.
The FSB wants the newly created Financial Conduct Authority to be given the power to investigate and recommend policies which focus on increasing competition within the banking sector. It is also calling for the government to ensure that small and new banks are not penalised under new regulations that force them to hold more capital.
FSB national chairman John Walker said that the lack of competition in the banking sector was giving small businesses less choice and making finance more expensive.
He added: "Even though Santander now has a small percentage of the small business banking market, the main four banks actually hold a larger share of the market than they did 10 years ago. The lack of competition in the sector is the most important thing that the Independent Commission on Banking should focus on in its full report in September."
Walker said that allowing new or specialised banks to buy branches from state-owned banks would boost competition on the high street, helping to drive down costs and give small businesses a fairer deal.
Figures published in May showed that banks under the government’s Merlin agreement had missed SME lending targets by more than £2bn.