2022 Magnet Ban "In-Scope" Products vs Injuries 

Intro: The Grand NEISS Database

The NEISS database, or the National Electronic Injury Surveillance System, is a system started by the U.S. Consumer Product Safety Commission (CPSC) in the 1970's to collect data on consumer product-related injuries treated in emergency departments in the United States. Unlike collections of anecdotal injury reports, the data and methodology used by NEISS are robust and reliable enough to make accurate inferences about the larger population. It's a vital tool for monitoring and analyzing product-related injuries in the U.S., enabling both regulatory agencies, and manufacturers to take proactive measures to ensure consumer safety, while also allowing the public to hold them accountable.

Determining with absolute certainty which NEISS injuries are attributable to a specific product is challenging due to several factors. The data in NEISS often relies on patients' subjective accounts, which might lack precision or be influenced by recall biases. Injuries can arise from a mix of factors, including external conditions, user behavior, or the involvement of similar or modified products. Additionally, the granularity of NEISS data might not always suffice to pinpoint a product's role in every injury. Factors like user modifications, delayed injury manifestations, and input error, further complicate the definitive attribution of injuries to specific products.

Navigate the NEISS's nuances naturally requires a systematic approach to classify injuries tied to specific products or categories. When delineating injuries linked to a product, researches must consider two fundamental questions:

"Are they doing what they say?" - Aka "Is their injury data actually the result of the classification criteria presented?" To ensure that the described method was actually used, it should yield reproducible results that align with the classification standards claimed.

"Are the criteria reasonable?" - Aka "Is the method for identifying injuries reasonably linked to the product?" The strategy should rest on sound principles, logically ensuring accurate injury identification while reducing misattribution or oversights.

In the CPSC's 2022 Magnet Set Rulemaking, which restricts American adults from acquiring high powered magnets, the agency failed on both metrics of scientific integrity. Notably, this marks the CPSC's second ban on these magnets for adults; the initial ban was implemented in 2015 but repealed in 2016. This guide explains how any layman can easily confirm that CPSC neither used valid reasonable criteria, nor adhered to their own claimed criteria from the outset.

1. One Elephant Exits, Two Bombshells Enter

The Pitfall of Over-Inclusion: A case study of CPSC's 2015 "Elephant in the Room"

Imagine if a city tried to ban pit bulls, based on bites from dogs that were "large" and/or have "short tails". While not likely to be precise or accurate, this method of counting pit bull injuries counted for 21% of all dog bites. This is parallels to the main criticism of the Consumer Product Safety Commission's (CPSC) 2015 Safety Standard for Magnet Sets ("2015 Ban" or "Old Ban): injury criteria were based on vague characteristics of magnets that were round and/or small. This ban was repealed in 2016 for being "arbitrary and capricious."

The Expansion of Scope in the 2022 Ban

Now, imagine this city banned pit bulls again, this time ascribing a disproportionately high percentage of dog bites to pit bulls by categorizing all "unknown" breed bites as pit bull bites, even though a great majority of bite reports in this city don't list a breed and are "Unknown". This is logic of the of the 2022 Safety Standard for Magnets ("2022 Ban" or "New Ban"), which attributes 82% of all magnet incidents to 'in-scope' ingestions, largely due to the inclusion of incidents involving "unknown" type magnets.

Data Inconsistencies and Suspected Manipulation

In the same hypothetical, the city's doesn't appear to be following their own stated method counting pit pull injuries. Severe data discrepancies exist between public records and the stated methodology for identifying in-scope injuries in the 2022 Ban. These timing of inconsistencies appear to exaggerate the success of the repealed 2015 ban, and underreport recent record high injuries that the CPSC may have indirectly facilitated.

2. Stated In-Scope Products vs In-Scope Injuries, 2022 vs 2015

The CPSC presupposes a full and direct correlation between 'in-scope' products and 'in-scope' injuries: "the benefits of the magnet rule would be the reduction in the societal costs of the injuries that would be prevented. Because the rule will eliminate from the market all magnet sets involved in the ingestion injuries described above, all injuries that would have occurred in the absence of a rule would be prevented." [LINK] Essentially, the Commission asserts that if a product within the ban's scope causes an injury, removing that product will prevent the injury. 

This assumption is problematic as it presumes perfect knowledge that is unattainable in practice. The NEISS database provides statistical estimates of ED incidents along with brief narrative descriptions, and there is no NEISS item category that specifically pertains to magnets alone. The CPSC's real obligation, therefore, is to ensure that "In-Scope" products are reasonably correlated with the "In-Scope" injuries. 

This begs the question: to what degree to "In-Scope" products targeted for regulation, actually correlate with "In-Scope" injuries attributed to those products? Here is what's claimed: 

Both the prior and current magnet ban share the same definition of a hazardous magnet, which is a magnet that can be swallowed (fits in the ASTM 1.25in diameter choke tube) and is powerful (flux index above 50 kG2mm2). 

The recent rulemaking primarily describes the same Spherical Rare Earth Magnet (SREM) Sets that were the subject product in the 2015 Ban:

"Subject magnet products are available in a variety of shapes, sizes (e.g., 2.5 mm, 3 mm, 5 mm), and number of magnets (1 to thousands). Subject magnet products often consist of numerous identical magnets, although some products include non-identical magnets, such as 2 or more different shapes. Subject magnet products commonly include magnets between 3 mm and 6 mm in size and consist of several hundred magnets." [LINK]

While the primary subject products described in both rules are similar, the scope of the products targeted has both broader and less defined under the current rule. For example, high powered hobby magnets from the hobby store would be banned as "amusement" magnets sold to consumers. While it's unclear if the same high powered hobby magnets sold in a hardware store, would be allowed for sale due to a commercial or industrial exception. 

But the most significant and concerning change, is the new scope of injuries counted against those subject products.

3. Broadly Uncertain Injury Data by Design. 

In the 2022 Ban, the CPSC has not been transparent with any of the 1,119 "In-Scope" incidents that contribute to In-Scope injury estimations from 2010-2021, as indicated in the 2022 rule. This prevents in-depth scrutiny of the integrity of their selection behavior. That is to say, the public has no way to confirm that the incidents blamed on the "In Scope" products, are actually chosen according to their own definitions, let alone that "In-Scope" incidents reasonably correlate with "In Scope" products to be banned.  

Despite the CPSC's lack of data transparency, we are still able to gleam useful insights peering through frosted glass by comparing Public NEISS data with the Annual Estimate of In-Scope incidents published in the 2022 Final Rule. The subsequent graph outlines the proportion of 'in-scope' magnet ingestions as a percentage of all magnet ingestions across both bans. Aside from the CPSC's "In Scope 2022" injury estimates, all data sets (MAGN, SREM) presented herein are the same ones described in the 2022 NEISS Analysis and the SREM Blueprint, which also offer reproduction instructions.

The 2010-2013 period presents a notable intersection, as it encompasses the date range that both the old ban and current ban evaluated for 'in-scope' injuries. Considering the similar scope of magnets banned, a conspicuous disparity emerges when analyzing the 'in-scope' injuries within this timeframe. Simply put, majority of all magnet ingestions are considered to be "In Scope" by the 2022 ban due to the inclusion of "unidentifiable" magnets. Of approximately 2,780 MAGN incidents per year from 2010-2013:

• 2015 Ban Injury Criteria: 21% of all Magnet incidents are In Scope. Approximately 580/year according to CPSC [See Pg 134, Table 2 of 2014 Final Rule].

• 2022 Ban Injury Criteria: 82% of all Magnet incidents are In Scope. Approximately 2,275/year according to CPSC [Table 5 of 2022 Final Rule].

The imprecision arising from the inclusion of the entire "unidentified" category of magnets, [See Table 3 of 2022 Final Rule] guarantees a degree of overestimation even greater than the faults that led to the overturn of the 2015 Ban. Where the Old Ban was a net cast in the direction of relevant injuries and identified one fifth of all magnet ingestions, the 2022 Ban happens to casts blame on the inverse, and attributes 'in-scope' injuries to all magnet ingestions, except for one fifth. 

Whether or not we are witnessing the acme of rulemaking intellectual inconsistencies within the CPSC, can be judged by anyone with an easily replicable look at public NEISS data.

4. Evidence of Cherry Picking 

If the first new Elephant in the Room of the CPSC's rulemaking is the broad inclusion of all 'unknown' magnet injuries to feed into their societal cost model. The second is the fact the compelling evidence that CPSC isn't adhering their own stated method of identifying 'in-scope' injuries. The CPSC appears to be engaging in selective data representation, rendering it's conclusions not only severely inconsistent, and it's motives questionable when lined up with their narratives.

According to the 2022 Ban's "In Scope" estimates, 80% of injuries are attributed to "Unknown" type magnets, while the remaining 20% arise from "Amusement and Jewelry". This means out of 25,000 "In Scope" injuries from 2010-2021, 20,000 were caused by unidentified magnets. Moreover, the "Master Set" [See Table 2 of 2022 Final Rule] of all magnet ingestions from this period excludes only 6.2% of cases (1,660 out of 26,600). Given the exceptionally small 6% slice of exclusions, it would be logical to expect the 'in-scope' category should largely align with overall magnet ingestion trends. However, the CPSC data contradicts this logical assumption.

Inconsistencies and Timing

The years 2014-2015 and 2017-2021 show a profound divergence between the "in-scope 2022" injury data and public NEISS data on magnet ingestions. For instance, 2020 showcased a sharp decline in 'in scope' injury data, reveals a substantial drop to 37% of all MAGN ingestions from 80% in 2016. This 43% decline is inexplicable and implies both dubious scenarios:

1. A sudden increase in specificity by physicians when reporting magnet ingestions in NEISS narratives, leading to a drastic reduction in either the "Unknown" category during the divergent years.

2. An arithmetically impossible increase in the excluded categories, which only constituted 6.2% of the CPSC's Master Set. 

The fact that in 2020, the SREM estimates rises to meet "In Scope 2022" at 37% of all MAGN ingestions, (2,200 total injuries) is quite ironic, as it pits the Old Ban directly with the New Ban. The only way for past CPSC and present CPSC to both be correct, is if there were precisely zero "Unknown" magnets consumed in 2020. Because the 2015 "In Scope" injury identification of Magnet Sets doesn't claim to include any "Unknown" magnets in the Old Ban, while the injuries "In Scope" of the New Ban includes all Unknown magnets, plus Magnet Sets, Jewelry, and all other Amusement magnets. 

Dissonant Narratives

The CPSC’s narrative posits that the 2015 ban on SREM was effective, with "ingestions plummeting during its implementation". However, the data does not support this assertion. In fact, as a portion of all magnet ingestions, SREM ingestions actually rose during 2015-2016 when the Old Ban was enforceable. 

Furthermore, the narrative told by the CPSC data is that magnet ingestion peaked in 2012, and then again in 2017. And that ingestions have stayed slower since then. Charting the absolute National Estimates for these data show another angle of the same chronicle, and tell a very different story:

When we examine the graph plotting the national estimates for MAGN, SREM, and "In Scope 2022", we see a different story than CPSC's narrative. The CPSC suggests that In-Scope injuries peaked in 2012, drastically dropped by the start of the First Ban in 2015, rebounded to a new high in 2017 post-repeal, and stabilized thereafter. However, this doesn't entirely align with the data. SREM and overall magnet ingestions seem to be currently at a peak, averaging 6,100 annually between 2020-2022—far greater than the 2009-2014 average of 2,500 per year. The data shows the CPSC has been significantly underreporting injuries, and in fact recent magnet ingestions have more that doubled the claims of the CPSC "In Scope 2022" estimates. [See Table 5 of 2022 Final Rule]

Implications and Speculations

The data patterns from 2014-2015 and 2017-2021 demonstrate clearly inconsistent selectivity in reporting. While it is conclusive that selective data presentation is at play, the intent behind such manipulation remains speculative. Possible motives could include:

• Exaggerating the effectiveness of the 2015 rule by under-counting "in-scope" injuries from 2014-2015.

• Evading criticism for the steep rise in magnet ingestions from 2017-2021 by not enforcing pre-existing safety laws.

• Avoiding the contrary evidence to the CPSC's claim that "warnings are not effective for magnets". As the recent prevalence of Magnet Set sellers on average provide noticeably weaker and less prevalent set of warnings than those from era of American SREM firms in 2009-2014.

The CPSC's apparent manipulation of data undermines not only its current rulemaking but also its credibility. The inconsistencies in "in-scope" injury data, particularly for the years after 2014, suggest a clear divergence from observable magnet ingestion trends. Thus, it raises questions about both the effectiveness and the necessity of the regulatory interventions being proposed.

5. Additional Methodological Concerns and Observations

The development and implementation of safety rules by regulatory agencies, such as the CPSC, require rigorous examination and careful selection of data. An analysis of the CPSC's methodology for the 2022 Magnet Ban raises several further concerns:

Incomplete Evaluation Period

A significant limitation in the CPSC's 2022 rulemaking is the absence of a control period prior to 2009 for evaluating the "in-scope" search strategy. The agency focused solely on the 2010-2021 period, despite the fact that some products under consideration did not exist before 2009. A broader examination period could offer a more comprehensive understanding of the issue, just as it would have enhanced the 2015 Magnet Set Ban.

Questioning the Necessity of Rulemaking

In the 2015 Magnet Set Ban rulemaking, the CPSC excluded 2013 injury data, acknowledging that their compliance actions had already decreased injuries, which could question the rule's relevance. According to the CPSC, the environment before these compliance actions represented a market untouched by CPSC intervention and served as the most accurate baseline for assessing the rule's impact. This exclusion suggests the CPSC recognized that compliance actions, not rulemaking, were the primary driver of safety improvements.

The CPSC stated: "Although the Directorate for Epidemiology's hazard analysis described injuries involving magnets from 2009 through December 2013, our analysis will be limited to 2009 through June 2012, before the request to stop sales, administrative actions, recalls, and public warnings ensued." [https://www.federalregister.gov/d/2014-23341/p-186]. Such an admission casts doubts on the necessity of additional regulations in the form of a ban.

Timing of Compliance Actions

While the CPSC states that compliance actions began in 2012, in reality, efforts against SREMs commenced in 2011. These efforts included field officers' deployments, information requests, and non-compliance letters to American magnet set firms by 2012. Evidence suggests that the decline in SREM injuries is attributable more to these compliance measures than to the 2015 ban, which was eventually repealed in 2016.

Unprecedented Ingestion Surge

The period from 2020-2022 has witnessed an unprecedented surge in SREM ingestions, averaging 2,000 cases per year. This rate is more than triple the average of 550 SREM ingestions per year for the period 2009-2014, before the 2015 ban was instituted. Moreover, each year since the onset of the COVID-19 pandemic has seen a higher relative SREM ingestion rate than any period prior to the 2015 ban.

6. Data Limitations

The methodology for the public NEISS data, including MAGN and SREM subsets, are detailed in MagnetSafety.org's 2022 NEISS Data analysis, with limitations discussed in section 5. 

"In Scope 2022" numbers are from [Table 5 of CPSC's 2022 Rule Briefing]. Since the 2014 data point was 'unreportable due to being under the NEISS reporting criteria', we use the same weight as 2015 (which had a similar NEISS incident count of 61, vs 62). It's likely that 1,200 incidents is an overestimate, and the real number exacerbates the divergence with public NEISS data. We can't know for sure until the CPSC releases the actual In-Scope incident list.

Comparison of MAGN and CPSC's 2022 "Master Set" of Magnet Ingestions

The CPSC's "master met" of magnet ingestions [described on pg 172 of 2022 Rule Briefing] is a subset of the more comprehensive MAGN list. That is to say, every item on the CPSC's "master list" is also on the more inclusive MAGN dataset. The difference is that MAGN set also includes aspirations (which present subsequent risk of ingestion, and are occasionally mislabeled ingestions), and is created from matches the text "MAGN" instead of "MAGNET". In depth rationale for MAGN data design can be found in in Section 4 of the MagnetSafety.org NEISS Analysis.

7. Conclusion, and CPSC's 2015 Magnet Ban For Context

In conclusion, a closer look at CPSC's 'In Scope' injury data raises serious concerns about the methodology and data selection practices underlying the CPSC's 2022 ban on high-powered magnets. The expanded scope of "in-scope" injuries to include all unknown magnet types essentially guarantees an overestimation of injuries attributable to the products targeted by the ban. More troublingly, the injury data presented fall far beyond the field of feasibility, and appears to selectively manipulated in ways that exaggerate the effectiveness of the repealed 2015 ban, while underreporting the recent surge in magnet ingestions that the CPSC may have indirectly facilitated through lax enforcement.

Understanding the 2015 Magnet Ban offers context. While the CPSC's 2015 methodology for identifying SREM injuries was flawed, its approach in 2022 appears to have regressed significantly further, seemingly neglecting the actual correlation between "in-scope" injuries and the products targeted for removal. The CPSC’s methodology from 2015 lacked aimed to identify incidents involving SREMs by selecting those that possibly or definitively involved high-powered magnet sets. The criteria for counting injuries for the 2015 Ban were imprecise, likely overestimating the number of genuine "in-scope" injuries. But in comparison, the 'in scope' criteria in 2022 more than quadruples 'in-scope' injuries, relative to all magnet ingestions. Despite evident methodological issues, the CPSC assumed a complete correlation between the scope of targeted SREM products and the reported "in-scope" injuries, overlooking the potential discrepancies in data in both 2015 and 2022.

Delving into the CPSC’s SREM classification, we anchored our analysis on detailed data from the 72 'in-scope' incidents of the 2012 rulemaking for the 2015 final rule. However, the 2022 magnet rule is bereft of such granular information. Despite this opacity, the inconsistencies in the CPSC's 2022 injury data peek brightly through the mist, due to the severity of apparent data manipulation. The preservation of data integrity demands clear categorization of incidents, and if the CPSC stands by its data, there’s no rationale behind obscuring the in-scope injury list from the public.

Ultimately, the inconsistencies and probable data cherry-picking practices documented herein undermine confidence in both the rationale behind the 2022 ban and the CPSC's credibility as an impartial regulatory agency acting in the interests of consumer safety. Before imposing sweeping prohibitions that stand to disrupt entire industries, the CPSC must provide full transparency into its data selection methodology and address the questionable patterns identified in this analysis. Only with methodological rigor and intellectual honesty can effective, evidence-based safety regulations emerge. Without timely reforms, the 2022 magnet ban might only be retracing the over-ambitious paths of its short-lived ancestor. This trajectory is further troubled by the CPSC's inadvertent anti-consumer maneuvers, pushing most hazardous magnets into international realms they cannot regulate.

Comments, or suggestions are welcomed. Email: outreach (@) magnetsafety.org

Sign up for notifications here.