It was a faustian bargain—and it certainly made editors at National Public Radio squirm.

The deal was this: NPR, along with a select group of media outlets, would get a briefing about an upcoming announcement by the U.S. Food and Drug Administration a day before anyone else. But in exchange for the scoop, NPR would have to abandon its reportorial independence. The FDA would dictate whom NPR’s reporter could and couldn’t interview.

“My editors are uncomfortable with the condition that we cannot seek reaction,” NPR reporter Rob Stein wrote back to the government officials offering the deal. Stein asked for a little bit of leeway to do some independent reporting but was turned down flat. Take the deal or leave it.

NPR took the deal. “I’ll be at the briefing,” Stein wrote.

Later that day in April 2014, Stein—along with reporters from more than a dozen other top-tier media organizations, including CBS, NBC, CNN, theWashington Post, the Wall Street Journal and the New York Times—showed up at a federal building to get his reward. Every single journalist present had agreed not to ask any questions of sources not approved by the government until given the go-ahead.

“I think embargoes that attempt to control sourcing are dangerous because they limit the role of the reporter whose job it is to do a full look at a subject,” says New York Times former public editor Margaret Sullivan. “It’s really inappropriate for a source to be telling a journalist whom he or she can and can’t talk to.” Ivan Oransky, distinguished writer in residence at New York University’s Journalism Institute and founder of the Embargo Watch weblog, agrees: “I think it’s deeply wrong.”

This kind of deal offered by the FDA—known as a close-hold embargo—is an increasingly important tool used by scientific and government agencies to control the behavior of the science press. Or so it seems. It is impossible to tell for sure because it is happening almost entirely behind the scenes. We only know about the FDA deal because of a wayward sentence inserted by an editor at the New York Times. But for that breach of secrecy, nobody outside the small clique of government officials and trusted reporters would have known that the journalists covering the agency had given up their right to do independent reporting.

Documents obtained by Scientific American through Freedom of Information Act requests now paint a disturbing picture of the tactics that are used to control the science press. For example, the FDA assures the public that it is committed to transparency, but the documents show that, privately, the agency denies many reporters access—including ones from major outlets such as Fox News—and even deceives them with half-truths to handicap them in their pursuit of a story. At the same time, the FDA cultivates a coterie of journalists whom it keeps in line with threats. And the agency has made it a practice to demand total control over whom reporters can and can’t talk to until after the news has broken, deaf to protests by journalistic associations and media ethicists and in violation of its own written policies.

By using close-hold embargoes and other methods, the FDA, like other sources of scientific information, are gaining control of journalists who are supposed to keep an eye on those institutions. The watchdogs are being turned into lapdogs. “Journalists have ceded the power to the scientific establishment,” says Vincent Kiernan, a science journalist and dean at George Mason University. “I think it’s interesting and somewhat inexplicable, knowing journalists in general as being people who don’t like ceding power.”

The press corps is primed for manipulation by a convention that goes back decades: the embargo. The embargo is a back-room deal between journalists and the people they cover—their sources. A source grants the journalist access on condition that he or she cannot publish before an agreed-on date and time.

A surprisingly large proportion of science and health stories are the product of embargoes. Most of the major science journals offer reporters advance copies of upcoming articles—and the contact information of the authors—in return for agreeing not to run with the story until the embargo expires. These embargoes set the weekly rhythm of science coverage: On Monday afternoon, you may see a bunch of stories about the Proceedings of the National Academy of Sciences USA published almost simultaneously. Tuesday, it’s the Journal of the American Medical Association. On Wednesday, it’s Nature and the New England Journal of Medicine. Science stories appear on Thursday. Other institutions have also adopted the embargo system. Federal institutions, especially the ones science and health journalists report on, have as well. Embargoes are the reason that stories about the National Laboratories, the National Institutes of Health and other organizations often tend to break at the precisely same time.

Embargoes were first embraced by science reporters in the 1920s, in part because they take the pressure off. After all, when everybody agrees to publish their stories simultaneously, a reporter can spend extra time researching and writing a story without fear of being scooped. “[Embargoes] were created at the behest of journalists,” says Kiernan, who has written a book, Embargoed Science, about scientific embargoes. “Scientists had to be convinced to go along.” But scientific institutions soon realized that embargoes could be used to manipulate the timing and, to a lesser extent, the nature of press coverage. The result is a system whereby scientific institutions increasingly control the press corps. “They’ve gotten the upper hand in this relationship, and journalists have never taken it back,” Kiernan says.

The embargo system is such an established institution in science journalism that few reporters complain or even think about its darker implications, at least until they themselves feel slighted. This January the California Institute of Technology was sitting on a great story: researchers there had evidence of a new giant planet—Planet Nine—in the outer reaches of our solar system. The Caltech press office decided to give only a dozen reporters, including Scientific American‘s Michael Lemonick, early access to the scientists and their study. When the news broke, the rest of the scientific journalism community was left scrambling. “Apart from the chosen 12, those working to news deadlines were denied the opportunity to speak to the researchers, obtain independent viewpoints or have time to properly digest the published research paper,” complained BBC reporter Pallab Ghosh about Caltech’s “inappropriate” favoritism in an open letter to the World Federation of Science Journalists.

When asked about why Caltech chose to release the news only to a select group of reporters, Farnaz Khadem, Caltech’s head of communications, stated that she is committed to being “fair and transparent” about how and when Caltech shares news with journalists. She then refused to talk about the Planet Nine incident or embargoes or press strategy, and she would not grant access to anyone at Caltech who might talk about such matters. As a consequence, it is hard to know for certain why Caltech decided to share the news with only a select group of reporters. But it is not hard to guess why journalists such as Ghosh were excluded. “It wasn’t that they were not good enough or not liked enough,” Kiernan speculates. “There was a real effort here to control things, making sure that the elite of the elite covered this story and covered it in a certain way, which would then shape the coverage of all other journalists. It’s very clearly a control effort.”

Caltech is not the only institution that steers coverage by briefing a very small subset of reporters. (As I was writing this piece, I received a note from a U.S. Air Force press officer offering a sneak preview of video footage being offered to “a select number of digital publications.”) For years the FDA has been cultivating a small group of journalists who are entrusted with advance notice of certain events while others are left out in the cold. But it was not the game of favorites that ignited a minor firestorm in the journalism community in January 2011—it was the introduction of the close-hold embargo.

Like a regular embargo, a close-hold embargo allows early access to information provided that attendees not publish before a set date and time. In this case, it was a sneak peek at rules about to be published regarding medical devices. But there was an additional condition: reporters were expressly forbidden from seeking outside comment. Journalists would have to give up any semblance of being able to do independent reporting on the matter before the embargo expired.

Even reporters who had been dealing with the FDA for years were incredulous. When one asked the agency’s press office if it really was forbidding communications with outside sources, Karen Riley, an official at the FDA, erased all doubt. “It goes without saying that the embargo means YOU CANNOT call around and get comment ahead of the 1 P.M. embargo,” she said in an e-mail.

“Actually it does need some saying, since this is a new version of a journalistic embargo,” wrote Oransky in his Embargo Watch blog. Without the ability to contact independent sources, he continued, “journalists become stenographers.” Kiernan echoes the sentiment: “[When] you can’t verify the information, you can’t get comment on the information. You have to just keep it among this group of people that I told you about, and you can’t use it elsewhere. In that situation, the journalist is allowing his or her reporting hands to be tied in a way that they’re not going to be anything, ultimately, other than a stenographer.”

The Association of Health Care Journalists (AHCJ), of which I am a member, publicly objected to the close-hold embargo, noting that it “will be a serious obstacle to good journalism. Reporters who want to be competitive on a story will essentially have to agree to write only what the FDA wants to tell the world, without analysis or outside commentary.” Faced by this opposition, the agency quickly backtracked. After a meeting with AHCJ leaders, Meghan Scott, then the agency’s acting associate commissioner for external affairs, wrote: “Prior to your inquiry, the FDA did not have a formal news embargo policy in place.” The FDA was now establishing new ground rules that “will better serve the media and the public.”

Initially published online in June 2011, the FDA’s new media policy officially killed the close-hold embargo: “A journalist may share embargoed material provided by the FDA with nonjournalists or third parties to obtain quotes or opinions prior to an embargo lift provided that the reporter secures agreement from the third party to uphold the embargo.” Due diligence would always be allowed, at least at the FDA.

Health and science journalists breathed a sigh of relief. The AHCJ expressed gratitude that the FDA had changed its tune, and Oransky’s Embargo Watch congratulated the agency for backing down: “For doing the right thing, the FDA has earned a spot on the Embargo Watch Honor Roll. Kudos.” And the FDA had cleared up the misunderstanding and affirmed that it was committed to “a culture of openness in its interaction with the news media and the public.”

In reality, there was no misunderstanding. The close-hold embargo had become part of the agency’s media strategy. It was here to stay—policy or no policy.

It is hard to tell when a close-hold embargo is afoot because, by its very nature, it is a secret that neither the reporters who have been given special access nor the scientific institution that sets up the deal wants to be revealed. The public hears about it only when a journalist chooses to reveal the information.

We have a few rare instances where journalists revealed that close-hold embargoes were being used by scientists and scientific institutions after 2011. In 2012 biologist Gilles-Eric Séralini and his colleagues published a dubious—later retracted and then republished—paper purportedly linking genetically modified foods to cancer in rats. They gave reporters early access under a close-hold embargo, quite likely to hamstring the reporters’ ability to explore gaping holes in the article, a situation science journalist Carl Zimmer described as “a rancid, corrupt way to report about science.” In 2014 the U.S. Chemical Safety and Hazard Investigation Board (also called the CSB) released a report to journalists under a close-hold embargo. When challenged, the then managing director of the CSB, Daniel Horowitz, told Oransky’s Embargo Watch that the close-hold embargo was used “on the theory that this would provide a more orderly process.” He then stated that the board was going to “drop the policy in its entirety for future reports.” Privately, however, a CSB public affairs specialist noted in an e-mail, “Frankly, I wish we did have more stenographers out there. Government agencies trying to control the information flow is an old story, but the other side of the story is that government agencies that do good work often have a difficult time getting their story told in an era of journalistic skepticism and partisan bickering and bureaucratic infighting.”

Also in 2014 the Harvard-Smithsonian Center for Astrophysics (CfA) used a close-hold embargo when it announced to a dozen reporters that researchers had discovered subtle signals of gravitational waves from the early universe. “You could only talk to other scientists who had seen the papers already; we didn’t want them shared unduly,” says Christine Pulliam, the media relations manager for CfA. Unfortunately, the list of approved scientists provided by CfA listed only theoreticians, not experimentalists—and only an experimentalist was likely to see the flaw that doomed the study. (The team was seeing the signature of cosmic dust, not gravitational waves.) “I felt like a fool, in retrospect,” says Lemonick, who, as one of a dozen or so chosen journalists, covered the story for Time(at the time, he was not on the staff of Scientific American).

The FDA, too, quietly held close-hold embargoed briefings, even though its official media policy forbids it. Without a source willing to talk, it is impossible to tell for sure when or why FDA started violating its own rules. A document from January 2014, however, describes the FDA’s strategy for getting media coverage of the launch of a new public health ad campaign. It lays out a plan for the agency to host a “media briefing for select, top-tier reporters who will have a major influence on coverage and public opinion of the campaigns.… Media who attend the briefing will be instructed that there is a strict, close-hold embargo that does not allow for contact with those outside of the FDA for comment on the campaign.”

Why? The document gives a glimpse: “Media coverage of the campaign is guaranteed; however, we want to ensure outlets provide quality coverage of the launch,” the document explains. “The media briefing will give us an opportunity to shape the news stories, conduct embargoed interviews with the major outlets ahead of the launch and give media outlets opportunities to prepare more in-depth coverage of the campaign launch.”

Ten reporters—from the New York Times, the Washington Post, USA Today, the Associated Press, Reuters, ABC, NBC, CNN and NPR—were invited to have their stories shaped. The day after the briefing, on February 4, everybody—except for the New York Times—ran with stories about the ad campaign. Independent comment was notably missing. Only NPR, which went live hours after the others, and CNN, in an update to its story midday, managed to get any reaction from anyone outside of the FDA. CBS plunked down an out-of-context quotation from the director of the Centers for Disease Control and Prevention, probably in hopes that readers wouldn’t notice that it was two months old. Nobody else seems to have tried to get anyone who could critique the ad campaign.

The result was a set of stories almost uniformly cleaving to the FDA’s party line, without a hint of a question about whether the ad campaign would be as ineffective as many other such campaigns. Not one of the media outlets said anything about the close-hold embargo. From the agency’s point of view, it was mission accomplished.

The FDA had a much harder task two months later. The agency was about to make public controversial new rules about electronic cigarettes. It was nearly impossible to keep the story from leaking out ahead of time; days before the new rules were going to be published in April 2014, rumors were flying. Reporters around the country could smell the story and began to e-mail the FDA’s press office with questions about the e-cigarette rules. The agency flacks would have to use all the powers at their disposal to control the flow of information.

“I’ve heard a number of rumors that the FDA will be releasing its proposed e-cigarette regulations on Monday,” Clara Ritger, then a reporter with the National Journal asked on Friday, April 18. “I wanted to see if I could confirm that? If that’s not accurate, do you have a timeline?” Stephanie Yao, then an FDA press officer, dodged the question: “The proposal is still in draft form and under review. As a matter of policy, the FDA does not share draft rules with outside groups while a rule is still under review.”

The fencing match was on. “Thank you for following up with the statement,” Ritger responded. “While I know the proposal is still in draft form and under review, for my planning purposes I wanted to find out when the proposed regulations will be coming out?”

“Have you subscribed to FDA press announcements?” Jenny Haliski, then another FDA press officer, wrote back on Monday. “The proposed rule itself will be published in the Federal Register.”

“Thanks for sending! I signed up,” Ritger responded. “The only other question I had was when the proposed regulations would come out, off the record, for planning purposes?”

Not even an offer of being off the record could get the agency to spill the beans. “The FDA can’t speculate on the timing of the proposed rule,” Haliski replied.

But this was a carefully crafted half-truth. There was no need to speculate. Haliski and others in the press office knew quite well not just that the rule was going to be published on Thursday, April 24, but also that there was going to be a close-hold embargoed briefing on Wednesday. It’s just that Ritger and the National Journal weren’t invited.

The invite list had been drafted days earlier, and, as usual, the briefing was limited to trusted journalists: the same outlets from the ad campaign briefing in February, with the addition of a few more, which included theWall Street Journal, the Boston Globe, the Los Angeles Times, Bloomberg News, Politico and the Congressional Quarterly. At the very same moment that the agency was discussing the embargoed briefing with some of their chosen reporters, anyone outside that small circle, like Ritger, was being thrown off the trail. Not even Fox News was allowed in.

Some within the FDA press office wondered why Fox was excluded, unlike the other major networks. “BTW, we noticed that Fox still wasn’t on the invite list,” Raquel Ortiz, then an FDA press officer, told Haliski.

“I have no national Fox reporter who had contacted me on this topic,” Haliski responded. “All reporters invited to the briefing needed to have covered tobacco regulatory issues before.”

Ortiz realized that this wasn’t an honest answer: “But they definitely cover FDA/CTP [Center for Tobacco Products] and tobacco stories—[a colleague has] seen them.”

“We don’t have a good contact for Fox,” Haliski insisted, rather lamely. A contact would not have been hard to find had they bothered to look. And, as chance would have it, the contact found them. Early the next morning, with plenty of time before the briefing, Fox’s senior national correspondent—John Roberts, one-time heir apparent to Dan Rather—contacted Haliski asking for access. “I’m aware that the FDA will likely come out with its deeming rule regarding e-cigarettes in the next week or so. I’d like to have a story ready to go for the day (holding to any embargo),” he wrote. “Can we make that happen?”

“Hi, John, Have you subscribed to FDA press announcements?” Access denied.

“I was particularly troubled by it because I was the medical correspondent for CBS Evening News for a couple of years, and I had a very good relationship with the FDA and everybody there,” says Roberts, who found out he was excluded after the other correspondents’ stories came out. “I was told by these folks that Fox news wasn’t invited because of ‘past experiences with Fox.’”

A little after noon on Wednesday, April 23, the briefing went on as scheduled. All the reporters present understood the terms, as announced: “As discussed, under this embargo you will not be able to reach out to third parties for comment on this announcement. We are providing you with a preview of the information with this understanding.” But by 2:30 P.M., the close-hold embargo was already fraying at the edges. FDA officials apparently got wind that a reporter was trying to talk to a member of Congress about the new rules. Even though it was not clear that this was a breach of the embargo—the interview was scheduled for after the embargo expired, and the reporter presumably did not share any crucial information ahead of time—it was bending the close-hold rules, and the FDA was livid. Within half an hour, FDA’s Jefferson had fired off an angry e-mail to the close-hold journalists.

“It has been brought to our attention that there has already been a break in the embargo…. Third-party outreach of any kind was and is not permitted for this announcement. Everyone who participated agreed to this,” she wrote. “Moving forward, we will no longer consider embargoed briefings for news media if reporters are not willing to abide by the terms an embargo…. We take this matter very seriously, and as a consequence any individuals who violated the embargo will be excluded from future embargoed briefings with the agency.” Violate the rules, even in spirit, and you’ll be left out in the cold with the rest.

The denials flew in. “This is very frustrating as someone who has consistently played by the rules and has covered CTP/FDA for years to be lumped in with a group of reporters that cannot respect your requests not to reach out to third parties,” insisted then AP reporter Michael Felberbaum. “I have of course always advocated that you work more closely with reporters like myself who clearly understand and cover this area consistently instead of reporters who are just assigned to handle on a whim.”

But despite the scare about a breach, the secrecy held. When the embargo expired and the early news stories went online, the FDA had little to complain about; the embargo had worked once again to shape coverage. Felberbaum’s piece, for example, quoted Margaret Hamburg, then head of the FDA, and Mitch Zeller, the head of the agency’s CTP, but nobody else. Even after he updated his piece later in the day to get some outside comments, there was little hint of how controversial the new rules were. Members of the tobacco industry were generally unhappy with increased federal regulation of their business, while antitobacco advocates tended to argue that the new regulations were far too weak and took way too long to promulgate. And there was no mention, in Felberbaum’s article, at least, that the agency had tried to regulate e-cigarettes several years earlier but was slapped down with a stinging rebuke from the U.S. District Court for the District of Columbia. (When asked about his work for the AP, Felberbaum—who has since quit his job as a reporter to become an FDA press officer—said, “I’m not really sure whether I’m comfortable discussing that at this point.”)

Some of the other outlets, like NPR, injected a little more nuance into their pieces, despite the restrictions, by doing additional reporting after the embargo expired. (In a statement, NPR said that agreeing to the FDA’s conditions was not a violation of ethics guidelines and “in no way influenced which other voices or ideas were included in the coverage.”) Still, even those pieces did not stray far from the key messages that the agency wanted to get across. Again the FDA found little to complain about. Except for one little thing.

Of all the media outlets, the New York Times was the only one to mention the close-hold embargo: “FDA officials gave journalists an outline of the new rules on Wednesday but required that they not talk to industry or public health groups until after Thursday’s formal release of the document.” (“I felt like I wanted to be clear with readers,” Sabrina Tavernise, the author of the story, later told Sullivan, the New York Times‘ public editor at the time. “Usually you would have reaction in a story like this, but in this case, there wasn’t going to be any.”)

The FDA was not pleased that the omertà had been broken. “I have to say while I generally reserve my editorial comments, I was a little surprised by the tone of your article and the swipe you took at the embargo in the paper—when after combing through the coverage no one else felt the need to do so in quite that way,” the FDA’s Jefferson upbraided Tavernise in an e-mail. “To be clear, this is me taking stuff personally when I know I shouldn’t, but I thought we had a better working relationship than this…. I never expect totally positive coverage as our policies are controversial and complex, but at least more neutral and slightly less editorialized. Simply put, bummer. Off to deal with a pissed Fox News reporter.”

Tavernise promptly apologized. “Geez, sorry about the embargo thing. Editors were asking why we didn’t get to see it so I was asked to put a line in to explain,” she wrote. (Tavernise declined to comment for this article; Celia Dugger, one of the New York Times editors who handled the piece, said via e-mail: “As to the decision to describe the conditions of the embargo in the story, Sabrina and I talked it over and agreed it was best to include them.”)

The FDA was not pleased that the secret of the close-hold embargo was out, and the excluded press was confused and angry. “In this particular instance, it struck me as very strange,” says Fox’s Roberts. “It was a government agency picking and choosing who it was going to talk to on a matter of public policy, and then the fact that I had a longstanding relationship with the FDA that, with this new administration, didn’t seem to matter.”

Oransky complained again on Embargo Watch about the FDA’s attempts to turn journalists “into stenographers.” Sullivan asked a few pointed questions of Jefferson, who, in Sullivan’s words, insisted that the FDA’s intent was “not to be manipulative but to give reporters early access to a complicated news development” and noted, in passing, that Tavernise had not objected to the terms of the close-hold embargo. But the damage was short-lived. Very little came of the complaints; Sullivan said that she would “like to see the Times push back—hard—against such restrictions in every instance and be prepared to walk away from the story if need be,” but there is no evidence of any substantial pushback by anyone.

The two-tiered system of outsiders and insiders that undergirds the close-hold policy is also still enforced. Major press outlets such as Scientific American and Agence France-Presse have written to the FDA to complain about being excluded but have not received any satisfaction from the agency. Months after the e-cigarette affair and following a different FDA story about food labeling that insiders had early access to, Time magazine complained about its lack of access to a select-press-only phone call. “Time was not included … (they weren’t even on my radar to be honest with you), but we handled all their queries” the day after the call, then FDA press officer Jennifer Corbett Dooren wrote.

Absent any indications from the agency, it is anyone’s guess whether the close-hold embargo is still in use at the FDA and, if so, how frequently. Unfortunately, the FDA refused to answer any questions. Because I am suing the agency for access to documents about embargo practices at the FDA, the press office, in a statement that failed to answer any specific questions, said that news embargoes “allow reporters time to develop their articles on complex matters in an informed, accurate way” and that its use of embargoes conforms to relevant government guidelines and best practices. The press office referred all questions to the FDA’s Office of the Chief Counsel, which did not supply answers.

Since the New York Times slip, no journalist covering the agency has openly mentioned being subject to such restrictions. Scientific Americanmade a significant effort to contact many of the reporters believed to have agreed to an FDA close-hold embargo—including the AP’s Felberbaum, the Times‘ Tavernise, NPR’s Stein, and other reporters from Reuters, USA Today and the LA Times. None could shed any light on the issue. Some explicitly refused to speak to Scientific American; some failed to return queries; two had no recollection of having ever agreed to a close-hold embargo, including Tom Burton, a Pulitzer Prize–winning Wall Street Journal reporter and the only one willing to answer questions. “I didn’t remember it at all, and [even] after you told me, I didn’t remember,” he said. As far as he knows, Burton added, such embargoes are rare.

No matter how rare it might be, there is documentary evidence of its happening multiple times, and each instance since 2011 is a violation of the FDA’s official media policy, which explicitly bans close-hold embargoes. This policy still stands, just as it did before the last close-hold embargo. The smart money says that the agency’s unofficial policy still stands, too—and the favoritism and close-hold embargoes continue. It is apparently too sweet an arrangement for the FDA simply to walk away.

Despite the difficulty of measuring the use of close-hold embargoes, Oransky and Kiernan and other embargo observers agree that they—and other variations of the embargo used to tighten control over the press—appear to be on the rise. And they have been cropping up in other fields of journalism, such as business journalism as well. “More and more sources, including government sources but also corporate sources, are interested in controlling the message, and this is one of the ways they’re trying to do it,” says the New York Times‘ Sullivan. “I think it should be resisted.”

As much blame as government and other institutions bear for attempting to control the press through such means, the primary responsibility lies with the journalists themselves. Even a close-hold embargo wouldn’t constrain a reporter without the reporter’s consent; the reporter can simply wait until the embargo expires and speak to outside sources, albeit at the cost of filing the story a little bit later.

Says Oransky: “We as journalists need to look inward a little bit and think about why all of us feel we absolutely have to publish something at embargo [expiration] when we don’t think we have the whole story?” Alas, Kiernan says, there isn’t any movement within the journalism community to change things: “I don’t know that journalists in general have taken a step back, [looking] from the 50,000-foot view to understand how their work is controlled and shaped by the embargo system.