You wouldn’t know it from the news, but there’s a major fossil-fuel spill ongoing in the North Sea off the coast of Scotland. A leak from a gas platform operated by the French energy company Total SA was first detected on March 25 and has been spilling around 7 million cu. ft. (200,000 cu m) of natural gas every day since. Of course gas, unlike oil, doesn’t have a devastating — or visual — effect on the marine environment, which is one reason the Elgin gas field, where the spill is taking place, hasn’t become as infamous as the Deepwater Horizon site in the Gulf of Mexico. But the leak is a disaster for the climate all the same; natural gas is mostly made up of methane, a greenhouse gas that has 25 times the warming power of carbon dioxide. Engineers working for Total estimate that it may take half a year to shut the leak, and if all of the methane released in that time reaches the atmosphere, the spill would approximate the annual global warming impact of putting 300,000 new cars on the road.

The Total leak is a reminder that natural gas — in the wrong place — can do very real damage to the environment, even if it does so invisibly. That might sound surprising because natural gas has been hyped as the clean fossil fuel, a replacement for coal power that’s better for the atmosphere. That and the vast new reserves of shale gas found in states like Montana and Pennsylvania have kept the price of natural gas low — the lowest it’s been in over a decade in the U.S. — and led to something of a boom time for the industry, enabling utilities to replace aging, polluting coal plants. “We, it turns out, are the Saudi Arabia of natural gas,” President Barack Obama said in a speech in January. “And developing it could power our cars and our homes and our factories in a cleaner and cheaper way.

The clean, green claim is more than just hype. Natural gas produces about half as much carbon dioxide as coal does when burned. But it’s not quite that simple. Some amount of methane will always leak away during the production of natural gas — particularly during the process of drilling or transporting via pipelines. And that means more warming. If enough methane leaks, natural gas loses its clean reputation — and the billions of dollars of new natural-gas infrastructure now being built could end up making the climate worse, not better. “We have to make sure that the climate benefits are real and continuous,” says Steven Hamburg, the Environmental Defense Fund’s (EDF) chief scientist, who has been studying methane-leakage rates.

That’s what makes a new study published in the April 9 Proceedings of the National Academy of Sciences (PNAS) so important — and so controversial. Researchers from Princeton University, Duke University, the Rochester Institute of Technology and EDF have distilled some of the existing data on methane leakage and calculated just how climate-friendly natural-gas electricity and natural-gas-powered vehicles are. And the answer is: it depends. On the whole, for both power plants and vehicles, natural gas beats conventional energy sources only if leakage rates are very low. The redline for power plants seems to be a leakage rate of 3.2%. As long as the share of methane that escapes remains below that level, natural gas beats coal. And since a 2009 estimate from the Environmental Protection Agency (EPA) pegs the average leakage rate at 2.4%, natural gas does seem to come out comfortably ahead — especially when you factor in the much lower levels of mercury and other traditional air pollutants emitted when power plants burn gas as compared with coal.

But if we replace gasoline cars with ones powered by natural gas in an effort to reduce oil consumption — as some like natural-gas tycoon T. Boone Pickens have urged — methane leakage would need to be kept below 1.6% to provide a meaningful climate benefit. For heavy-duty trucks run on diesel — a fuel that has a smaller carbon footprint than gasoline — methane leakage would have to be reduced by two-thirds below the EPA estimate of 2.4% to provide a climate benefit. “It’s a much steeper hill to climb,” says Hamburg.

There are, of course, many caveats in all this data. Even the EPA has admitted that its 2009 estimate of methane leakages is likely outdated and thus inaccurate, as the trade group America’s Natural Gas Alliance pointed out in a response to the PNAS paper. But there’s no guarantee that the actual methane-leakage rate is lower than the EPA estimate; it could be higher. In a controversial 2011 study, Cornell scientist Robert Howarth pegged the leakage rate at between 2.2% and 3.8%, while another study published this year in the Journal of Geophysical Research estimated that methane-leakage rates at a field in Colorado were closer to 4%, which would make natural gas worse than coal, at least over the short term. (While methane has a stronger warming effect than CO2, carbon dioxide lingers in the atmosphere far longer than methane.) “We can’t say, and I don’t think anyone can say, ‘Here is the absolute truth about natural gas compared to other fuels,’ because the emissions data is highly uncertain,” says Ramon Alvarez, a senior scientist at EDF and a lead author on the PNAS study. “At this point, we just have to go with the best information we have.

That’s why we need better information — and indeed, EDF is working with a number of industry partners to get a much more precise figure on just how much methane is escaping during the production of natural gas, especially unconventional shale gas. (That sets EDF apart from a number of other environmental groups — including the Sierra Club — that are waging political war on the shale-gas industry.) America may indeed be the Saudi Arabia of natural gas, and air-pollution regulations and other economic factors will almost certainly speed the shift from coal-fired power to natural-gas plants. But it would be a terrible mistake to make that transition without accounting for the risks to the climate — especially since reducing methane leaks is a lot easier than, say, taking the carbon out of coal. Just because methane is invisible doesn’t mean it’s harmless.