The NHS faces a decade-long savings drive, managers believe.
The warning from the NHS Confederation came on the day official workforce figures showed nursing posts had fallen by 1% in the past year.
It comes despite promises by ministers that the frontline would be protected during the efficiency drive.
The NHS has been told to find £20bn savings by 2015, but the confederation said another £20bn would have to be found in the five years after that too.
David Stout, NHS Confederation deputy chief executive, said: "The working assumption is that the NHS will be required to continue to produce significant savings beyond 2015. The initial £20bn is not the end game."
He added it would all depend what the funding settlement was post-2015, but with most chief executives expecting something similar to the last one it was fair to assume "another £20bn" would have to be found to cope with rising costs from the ageing population and cost of new technologies.
"These figures show the scale of the challenge the NHS faces. We need to be honest about the impact this will have and relay to the public the changes that will be required to NHS services in order to maintain the best care for patients."
'Just the start'
Professor Chris Ham, the chief executive of the King's Fund think-tank, said: "It promises to be an even more challenging period for the NHS than expected."
The BBC understands the extra savings have even been discussed in recent meetings between senior Department of Health figures and NHS finance chiefs.
But a government source said "this is at the upper end of our expectations, it depends on a range of different scenarios depending on funding assumptions and cost pressures".
Meanwhile, the figures released by the NHS Information Centre have raised questions about how the current productivity push is being carried out.
The aim was to look at new ways of working to save money, but unions said the data suggested patients were suffering.
Put bluntly, the idea that cutting hundreds of jobs from a hospital will not affect the care of patients is ludicrous”
End QuotePeter CarterRoyal College of Nursing
The figures showed the number of nurses employed fell by 3,500 in 2011 - 1% of the total.
Overall staff numbers fell by nearly 20,000 - 1.4% - to 1,350,000, but clinical professions such as GPs and consultants saw a rise in headcount.
The biggest falls were seen in support and managerial posts - something that was expected because the overhaul of the NHS has led to thousands of staff leaving primary care trusts, the management bodies which are being abolished.
Nonetheless, there are still a fifth more people working in the health service than in 2001. But unions believe the drop in nurse numbers is just the start.
Peter Carter, general secretary of the Royal College of Nursing, said the figures were "incredibly worrying" and the situation was only going to get worse.
He added: "Despite the rhetoric, we know that frontline jobs are not being protected and NHS trusts must stop making cuts in a quick fix attempt to save money.
"Put bluntly, the idea that cutting hundreds of jobs from a hospital will not affect the care of patients is ludicrous."