2022 NEISS Magnet Injury Data Observations
2003-2022 NEISS ED Magnet Ingestion Trends: Analysis and Step by Step Reproduction Tutorial
1. Magnet Metrics - Ingestions Near All Time High
The National Electronic Injury Surveillance System (NEISS) unveiled its 2022 data, spotlighting a continuing surge in incidents related to magnet ingestion and aspiration within the US. Deriving data from a diverse pool of over 100 hospitals, NEISS offers a steadfast and statistically significant sightline into the shifting trends since 2003. This analysis illuminates injury inclinations, underscoring the user-friendliness of the NEISS database, while also offering a penetrating perspective on the perplexing problem of magnet ingestion.
To convey the convenience of using publicly accessible NEISS data, we map out our data methodology in the form of a step-by-step tutorial, enabling effortless emulation of most depicted data in mere minutes. No lab coat required.
2. Dataset Definitions
There are four primary NEISS data sets we are evaluating. The later section 4. Methodology and Tutorial - Sans Labcoat, goes into more detail about the data sets, as well as providing reproduction instructions in an easily accessible manner.
ALL_Ingest - This dataset serves as a comprehensive compilation of all incidents in public NEISS data related to ingestion (Diagnosis code=41), and also includes aspirations (Diagnosis code=42). The rationale for including aspirations in "all ingestions" data set is explained below.
MAGN - Extracted from ALL_Ingest, this dataset is confined to incidents related to magnets. Identification is achieved through textual matches for "MAGN" within the incident narratives, devoid of any additional subjective filtering.
BATT - Functioning in a fashion similar to MAGN, this dataset identifies battery-related incidents within ALL_Ingest. Cases are selected based on the appearance of the text "BATT" in the narrative, offering a straightforward yet robust method for locating battery ingestion incidents.
SREM - Approximates "In Scope" injuries of the 2015 magnet ban that primarily targeted Spherical Rare Earth Magnets. This is set of incidents is based on CPSC's 2012 prevailing selection criteria based on the 72 in-scope NEISS incidents, described as identifying in-scope "High-Powered and/or Ball-Shaped Magnet Ingestions That Were Determined To Involve, or Possibly Involve, the Magnets of Interest". A step by step tutorial using the GPT Quorum method for subjective delineation, is provided in depth [here: CPSC's SREM Blueprint]
The MagnetSafety.org 2003-2022 NEISS Analysis spreadsheet shows each of these datasets. The spreadsheet can be accessed and downloaded free of charge here: [https://docs.google.com/spreadsheets/d/1DgAv9QOVzxC-3nfB5JpvCJBflv7zyk8ducwSkOqcYcY]. The graphs below present the annual national estimates based on the sum of weights for these four datasets, which are also documented in the "Analysis" tab of the spreadsheet. The sheet titled "ALL_Ingest" encompasses both MAGN and BATT data, marked with boolean flags in column AA and BB respectively. The process and results for specifically identifying a SREM, are housed in the "MAGN_Only" tab.
3. Data Overview and Observations
All Foreign Body Ingestions: In 2022, emergency room visits due to foreign body ingestions and aspirations rose to an estimated 114,200 cases. This signifies a 6.5% year-over-year growth and a 67% increment from 2003. In the same 2003-2022 period, US Population only increased 15%. Notably, the 95% confidence interval has been continually increasing in variance over time, most recently ranging between 80,000 to 148,400 in 2022.
Magnet Ingestion Trends: Post the 2017-2022 CPSC SREM Ban, daily magnet ingestions averaged 14, a sharp rise from the 6.9 average between 2009-2014, prior to the initial 2015-2016 SREM ban. Despite a decrease from 6,800 in 2021 to 6,500 magnet ingestion cases in 2022, recent magnet ingestions remain considerably above the prior peaks of 3,000 (2007) and 3,400 (2012).
SREM Ingestion Trends: Post-Covid data from 2020-2022 reveals an average of 2,000 SREM ingestions annually. This is nearly four times the yearly average of 540 cases from 2009-2014.
Battery Ingestions: These have grown considerably from 3,900 in 2006 to 9,900 in 2022. Although BATT incidents in 2022 were 50% greater than MAGN incidents, MAGN's growth rate over nearly two decades was 3.5x, compared to BATT's 2.5x. Batteries are contextually similar to magnets, in that they are also significantly more dangerous than typical foreign body ingestions.
Relative to All Ingestions: Before SREMs were introduced in 2009, magnet ingestions had already peaked in 2007. Using the CPSC's criteria for "round and/or strong," it's evident that magnetic ingestions fitting this description were more common before Spherical Rare Earth Magnets (SREMs) came into play. For 2006-2008, 1.0% of all ingestions were SREM, and 3.1% were MAGN. By 2009-2011, these figures shifted to 0.6% and 2.7%, respectively. The data suggests more ingestions fit the CPSC's criteria before the actual product introduction, raising questions regarding the CPSC's identification process.
Coincidentally, CPSC compliance efforts against SREM's started in 2011 in the form of field officers deployment, and escalated sending information requests, and non-compliance letters to American magnet set firms in 2012. Compliance actions appeared to continue up to the 2015 Magnet Set Ban [according to CPSC Magnets Enforcement Page, archived Jul 2021]. New Notices of violation did not begin again until 2021 [via CPSC Magnets Page archived Aug 2021]
Relative to All Magnet Ingestions: Contrary to expectations the SREM ban, that was enacted in 2015 and repealed in 2016, didn't reduce relative SREM ingestions. During the ban's active years (2015-2016), the percentage of SREM ingestions rose while total magnet ingestions decreased. Historically, the ratio of SREM to MAGN was highest in 2008, preceding the introduction of high powered magnet sets to market. While less than 2% of MAGN incidents in 2003 were described as round or high-powered magnets, this proportion grew to 19% in 2004 and further to 27% in 2005. Intriguingly, the first magnet ingestion articles emerged in 2004 [LINK], and 2005 coincides with the expiration of the main NdFeB magnet patent (US No. 4,588,439). The all time high SREM to MAGN ratio occurred in 2008, before the introduction of 5mm rare earth magnet sets, matched by the 2020 figures, aligning with the advent of Covid.
4. Methodology and Tutorial - Sans Labcoat
The aim of this study is not merely to articulate data methodology, but to demonstrate the ease and straightforwardness of utilizing publicly available data from the National Electronic Injury Surveillance System. Once NEISS data is consolidated into a single spreadsheet (ALL_Ingest), most datasets can be generated in a mere span of 3 minutes, namely ALL_Ingest, MAGN, and BATT using software such as Excel or Google Sheets. With the exclusion of the SREM data set, the visual representations provided in the preceding sections are unembellished renditions of these NEISS data sets, free from any subjective inclusions or exclusions.
ALL_Ingest - Compiling
ALL_Ingest serves as the comprehensive catalogue of all recorded incidents related to Ingestion (Diagnosis Code 41) and Aspiration (Diagnosis Code 42) within the NEISS database. Although primarily targeting Ingestions, three rationale underscore the inclusion of Aspiration incidents within this data set:
Ingestions are occasionally mislabeled with the Aspiration diagnosis code.
Aspiration was manifest in the 72 incidents identified as SREM by the CPSC in their 2012 Magnet Set NPR. (Note that Aspirations do not appear to be included in the set of NEISS incidents considered by CPSC in 2022 Magnet Ban Rulemaking.)
Magnets adhered to nasal passages possess a subsequent risk of ingestion, a known hazard pattern associated with Spherical Rare Earth Magnets (SREMs).
To aggregate the ALL_Ingest data set, one may follow these procedural steps outlined below:
Accessing NEISS Database: Navigate to the NEISS Query Builder, select your affiliation and commence query construction.
Filtering by Treatment Dates: Opt for the "Most Recent 10 Years" to target data from 2013-2022.
Filtering by Diagnosis: Confine your search to Diagnosis Codes 41 and 42.
Output Customization: Organize results by year and select Coefficient of Variation, 95% confidence Interval of Estimate, Number of Cases.
Data Download: Obtain the data in MS-Excel format for further analysis.
For capturing data ranges that extend over a decade, repeat steps 2-5 with earlier timeframes in five-year blocks, and concatenate them manually or with a free app. Alternatively, the pre-compiled spreadsheet that includes NEISS data with Diagnosis 41 or 42, is available here, and is identical to the output of the steps above for 2003-2022.
ALL_Ingest - National Estimates
To segregate yearly incidents and procure Annual National Estimates for Ingestion and Aspiration incidents in under 3 minutes:
1. Craft a new column for the year of each incident using a formula: Within All Ingest, insert a new column after Column B "Treatment Date", called "Treatment Year". In cell C2 of the new column enter this formula to get the year from the date:
=YEAR(B2)
Fill this formula to cells below by double-clicking the lower right of cell C2. You may need to format the entire column as an integer if getting #name errors.
2. Index the years in an empty column: For example column AH, starting at 2013 in cell AH2.
3. Calculate national estimates by summing Weights: Title the next column "ALL Est" at the top. The sum of the *weights* for any set of NEISS incidents represent the national injury estimate for those incidents. To automatically count the weights per each year, use the following formula to seed in cell AI2, which works in both google sheets and excel:
=SUMIF(C:C,AH2,Z:Z)
This is assuming your index years start in cell AH2, and column Z contains the weights. Drag the formula down to the rest of the index years in AH. You'll know you're doing it right the output matches National Estimates from the NEISS query builder download screen.
MAGN - Identifying and Getting Estimates
MAGN catalogues magnet-related incidents in ALL_Ingest based on the mention of "MAGN", correlating closely with magnet ingestion. To see the MAGN incidents in the data provided, filter "ALL_Ingest" by MAGN=1. To acquire national estimates for incidents involving magnets in under 3 minutes:
1. Identify incidents with magnet mentions: Using Column AA as an example empty column, title it "MAGN" in the top cell (AA1). To automatically search for "MAGN" in the Narritive data, formula to seed in cell AA2 that works in both google sheets and excel, and drag down or double click to propagate:
=IF(ISNUMBER(FIND("MAGN",W2)),1,0)
This is assuming Column W is the Narrative column. The output will be '1' if magnets are mentioned. You can also use conditional formatting to color each MAGN=1 positive match for easier visualization.
2. Derive National Estimates by Year: The sum of the Weights of any set of NEISS incidents, are the national injury estimate for those incidents*. We'll use column AJ to output our estimates for MAGN incidents per year. Seed this formula in AJ2:
=SUMIFS(Z:Z,C:C, AH2,AA:AA, 1)
Where column Z are weights, col C is the Treatment Year, col AA is MAGN Positive, and AH2 is the index year in question. Drag the formula down to all index years. While rudimentary, this is an easily reproducible method that works very well for finding all incidents that mention magnets, and will deterministically produce the same result every time without the pitfalls of bias that come with subjective human filtering.
BATT - Identifying and Getting Estimates
In a similar vein, incidents with the descriptor "BATT" typically indicate battery ingestion. To see the MAGN incidents in the data provided, filter "ALL_Ingest" by BATT=1.
In under 3 minutes, here's how to discern national estimates related to batteries:
1. Spot Battery incidents in the narrative: Using Column AB as an example output column, enter title "BATT" in cell AB1. Check for "BATT" in narratives by seeding the following formula in AB2 and propagate down the whole data set:
=IF(ISNUMBER(FIND("BATT",W2)),1,0)
Conditional formatting the 1 or 0 output is optional to help visualization, but recommended. While there is a product category for batteries, the "BATT" text search is more inclusive.
2. Compute National Estimates by year: Sum the weights of all BATT incidents by copying and expanding this formula in AK2:
=SUMIFS(Z:Z,C:C, AH2,AB:AB, 1)
Where column Z are weights, col C is the Treatment Year, col AB is BATT Positive, and AH2 is the index year in question. Drag the formula down to all index years. While there is a product category (884) for batteries, the method of searching for BATT in narratives is more inclusive and comprehensive than a category search.
SREM - An AI Augmented Approach
Unlike the method for highlighting the MAGN and BATT incident sets, the method for identifying the SREM incidents is not deterministic, and is subjective. Since the CPSC's selection criteria employed was neither straightforward nor uniformly applied, using the GPT Quorum method, we emulate the prevailing patterns of CPSC's 2012 SREM selection method for "in-scope" products based on their 72 incidents involving high-powered and/or ball-shaped magnets.
These 72 incidents are important because they are the only publicly available selections for what CPSC considered "in-scope". The CPSC has not published the "In-Scope" incident list for the November 2014 Final Rule, January 2022 NPR, or October 2022 Final Rule on magnet sets. And thus, It is impossible to audit the consistency, validity or efficacy of the CPSC's "in-scope" selections for those.
Find the SREM Blueprint here:
The SREM Blueprint tutorial linked above details a step by step GPT-4 augmented process in identifying SREM, as well as how the prevailing criteria was determined despite inconsistencies. To project NEISS national estimates for SREM incidents, simply sum the Weights of all SREM incidents.
A generalized summary of the method for identifying SREM incidents is:
1. In a sheet of only MAGN incidents, pre-match any of the following text: POWER, RARE, MARB, BALL, BB, BEARING, BEAD, SPHER, ROUND, LIP, PIERCE, TONGUE, TOUNG.
2. Exclude matches if they aren't magnet sets. This is a subjective common sense step. Some examples of exclusions would be if a pre-match for "BALL" was referring to a steel ball. Or if the component that string matched was noted to be non-magnetic. Or if the string match was not relevant to being Round, Powerful or used around the mouth, like a match for "BB" due to the word "RUBBING".
3. For Uncensored data only, include matches that refer to a specific known brand of SREMs. This step does not apply to the Publicly available NEISS data, because brand names are redacted.
5. Dataset Limitations
All NEISS data - The limitations of the ALL_Ingest dataset are primarily those inherited from the design of NEISS database, and apply to all NEISS data:
Sampling: NEISS is based on a sample of hospitals, not all hospitals in the U.S. The data is then weighted to make national estimates. This means there's potential for sampling error. As noted by CPSC, estimates cannot be relied upon if:
Estimate is less than 1,200
Number of cases is less than 20
CV exceeds 33%
To calculate the CV for any set of NEISS incidents in a manner that takes into account the probabilities of selection, stratification, and weighting, see instructions on Pg.13 of the current NEISS Sample Design and Implementation Documentation.
Detail Limitations: NEISS reports often contain brief narratives which can lack sufficient detail to understand the complete context of the injury.
Public NEISS Data: The data available to the public may have reasonable redactions several reasons: including patient privacy, brand confidentiality, and ongoing CPSC investigations.
Accuracy and Consistency: As with any large-scale data collection effort, there can be inconsistencies and inaccuracies in the data due to differences in reporting practices among hospitals or mistakes during data entry.
Inpatient Data: NEISS mainly captures data from emergency departments, so injuries that are treated in other parts of the hospital or other medical facilities might not be recorded. This is why ICM estimates are also used for rulemaking.
MAGN - The simple text search for "MAGN" within NEISS narratives is effective and deterministic, but is not perfect. Searching for "MAGN" is more effective and encompassing than searching for the whole word "MAGNET", as it catches many misspellings, such as "MAGNATIC" (For example NEISS ID# 50331618, 120214592), "MAGNECTIC" (80210439), "MAGNATIX" (81051740), "MAGNTIC" (150145260, 200918403), "MAGNITIZED" (190205851), "MAGNENT" (201246351). Matching for "MAGN" does not catch all misspellings, such as "MAGENT" (200537546), and there is no additional filter for false positives such as "NON-MAGNETIC" (130564667), or unrelated words such as "MAGNESIUM" (180804422). From a manual reading of MAGN incidents, this simple deterministic method is at least 97% accurate in identifying incidents that may involve magnets.
BATT - Similarly, this simple text search for "BATT" within NEISS narratives that highly correlates with ingestions that involve batteries. It can catch a few misspellings of battery such as "BATTRY" (130953519), but is not immune to negative descriptors like "NOT A BATTERY", or unrelated words like "BATTLESHIP" (120942905). It also doesn't include misspellings of Battery that don't start with "BATT". Similar to MAGN, searching for "BATT" appears to be over 97% accurate in flagging Battery related incidents.
SREM - The SREM Identification Blueprint follows CPSC Epidemiology's 2012 analysis method for determining incidents that "possibly involve", or involve, high powered magnet sets, (https://www.federalregister.gov/documents/2012/09/04/2012-21608/safety-standard-for-magnet-sets).
While "SREM" set of incidents represents the CPSC's best effort to select incidents correlating with a high likelihood of involving the ingestion of Sets of Rare Earth Magnets, there is substantial evidence that shows the CPSC's SREM identification criteria is not especially effective at identifying SREMs.
For example, when the CPSC's SREM Identification Blueprint is applied to the three years 2006-2008 before SREMs were prevalent, there were 2,200 injuries from "in-scope" products described as round/high powered. This contrasts with the 1,700 estimated injuries from in-scope products in the 2009-2011. In other words, CPSC's SREM identification method identified a a larger prevalence of products involved in round or high powered magnet ingestion magnets in 2006-2008, before SREMs had hit the market.
Despite this, the CPSC assumed 100% correlation between the scope of the SREM products to be banned, and the injuries that were in-scope: "...the benefits of the magnet rule would be the reduction in the societal costs of the injuries that would be prevented. Because the rule will eliminate from the market all magnet sets involved in the ingestion injuries described above, all injuries that would have occurred in the absence of a rule would be prevented." [LINK]
Just as they did for the 2015 Magnet Ban Rulemaking, in development of 2022 rulemaking, CPSC also forgot to test the efficacy of their "in-scope" search strategy on a wider control period, namely before 2009 when the specific "in-scope" products to be banned didn't yet exist.
Submissions for Comments, Suggestions and Corrections are welcome. Please email outreach (@) magnetsafety.org.
Sign up for notifications here.