[Valerie Yoder]

Hi – you don’t need to include the default NPCR fields if you don’t require them from data contributors. I believe they were included in the tool to make it easier for those registries that do.
Your state’s user dictionary can just include those data items that you request from hospital registries or other data contributors, that are not in the NAACCR data dictionary.

[Valerie Yoder]

Sorry to get your hopes up, reading looks good but I forgot to check writing! It seems broken in 4.9 and 4.10. The temporary output csv file contains all of the patients & tumors, but the xml only contains the first patient, their first tumor, and a tumor from another patient down to rxTextSurgery. I don’t see anything obvious about why it stopped writing and why the tumor got placed under the wrong patient, it is correct in the temp CSV.

ex.
<Patient>
<Item>Patient 1 info</Item>
<Tumor>
<Item>Patient 1’s tumor info</Item>
</Tumor>
<Tumor>
<Item>Patient 2’s tumor info</Item>

Successfully wrote:
<Item naaccrId=”textDxProcPath”>9-9-16 HOSPITAL PATH-16-99999 BRAIN, TEST, TEXT: TEXT, WHO GRD 999. TEXT, BRAIN TMR: X/X TUMOR. XXX-9: TEXT. TEXT: TEXT, TEXT: TEXT</Item>
<Item naaccrId=”textStaging”>N/A</Item>
<Item naaccrId=”rxTextSurgery”>9-9-16 HOSPITAL: TEXT W/TEXT TEXT BY DR DOCTOR</Item> end of writing

next item to be written, is present in tmp csv rxTextRadiation:
9-9/9-9-16 HOSPITAL, DR DOCTOR: XXX BRAIN (9999 CGY), 99 FX’S, XXXX & 9MV

It successfully wrote some variables that were read with CDATA, so I don’t think that was the problem.

[Valerie Yoder]

Looks good to me, I don’t see any other problems at this time!

[Valerie Yoder]

Yes overall the fix is working for CDATA sections! Just one minor additional fix, when there are pairs of [] within the text, the second ] onward is consistently not read. It’s cut off in the temp csv and sas dataset.
Example xml:
<Item naaccrId=”rxTextChemo” naaccrNum=”2640″><![CDATA[1/10/2016 DrugB (Part1, Part2, & Part3) @ Facility w/ Dr. Name. [DrugA started in 1/2015, but DrugB regimen planned] 1/15/2017 Drugc @ Facility w/ Dr Name2]]></Item>

The resulting variable only contains:
1/10/2016 DrugB (Part1, Part2, & Part3) @ Facility w/ Dr. Name. [DrugA started in 1/2015, but DrugB regimen planned

I think this is likely because CDATA uses [], I didn’t have problems with any other special characters in the data I tested.

[Valerie Yoder]

The truncation issues seem to be fixed! I agree the macro files don’t need versions.

There’s a problem reading text fields that contain CDATA, they are not imported.
Example:
<Item naaccrId=”rxTextRadiation” naaccrNum=”2620″><![CDATA[1/1/18 HOSPITAL – DR X. O’EXAMPLE: SOMETEXT – MORETEXT & SOMEMORETEXT]]></Item>

[Valerie Yoder]

I tried the examples with Fabian’s JAR and I think it’s the best solution for SAS so far. It’s more straightforward to a user than the XML Mapper (I did not try tagsets). No special configuration was necessary which is great. I like the ability to specify a short list of variables to read, this is easier than what we currently have to do for flat files!

I found the speed very good – it took slightly less time to read one of our annual submission files in xml than it did to read the flat file (both v16), and about twice as long to write it back out. I read abstracts in SAS far more often than I write them, so the writing being slower doesn’t bother me. When I read a smaller full abstract file from a hospital (v16, converted), the speed difference was negligible.

Suggestions:
-Add ‘replace’ option to the tmp csv import step
-There are some truncation problems on import. With the submission file I observed this short list of variables imported up as $1. when they should be $2.-$4 (and therefore the truncated value was written out). tumorSizeSummary, tnmEditionNumber, tnmPathT, tnmPathN, tnmPathM, tnmPathStageGroup, tnmClinT, tnmClinN, tnmClinM, tnmClinStageGroup, radRegionalRxModality
When I read full abstracts there were 65 variables that were truncated such as addrCurrentCity imported as $14. instead of $50.

[Valerie Yoder]

I work in informatics at the Utah Cancer Registry and use SAS frequently so I am very interested in this topic. I will try out the code soon.

We read NAACCR files much more often than we write them, and we very rarely read all the variables in SAS. I think it could be reasonable to create a full mapping file for the community and recommend that they only keep the relevant variables- I can just delete variables from the .map file Fabian provided using Notepad++.
Even though writing from SAS is problematic, so much of our work and analysis goes through SAS that I don’t see us moving away from it for a long time. We would likely want a different tool to convert SAS output to NAACCR XML (like Seer Data Viewer? or something in Python, if its xml packages can make a valid file).

[Author]

Posts