Fabian Depry

Fabian Depry

Forum Replies Created

Viewing 15 posts - 1 through 15 (of 39 total)
  • Author
    Posts
  • in reply to: using SAS with NAACCR XML #13207
    AnonymousFabian Depry
    Moderator

    Lin,

    I found the issue and fixed the write macro. You can find the new version (7.3) on the release page of the Java NAACCR XML project (https://github.com/imsweb/naaccr-xml/releases/).

    I also updated/improved the NAACCR XML and SAS wiki page (https://github.com/imsweb/naaccr-xml/wiki/7:-NAACCR-XML-and-SAS).

    Let me know if this is still not working as you expect.

    • This reply was modified 6 days, 4 hours ago by AnonymousFabian Depry.
    in reply to: using SAS with NAACCR XML #13206
    AnonymousFabian Depry
    Moderator

    Hi Lin,

    There is a definitively a problem in the write macro and I think I know what it is.

    I am going to take a closer look today and release an updated version soon.

    in reply to: user dictionaries #13137
    AnonymousFabian Depry
    Moderator

    There is also some good information on this wiki page:

    https://github.com/imsweb/naaccr-xml/wiki/3:-NAACCR-XML-Dictionaries

    In particular, the page contains links to 3 free software that can be used to create dictionaries. Those applications provide a nice GUI, they will validate the content and write it nicely formatted. Trying to create a dictionary by hand is not recommended.

    in reply to: Trimming the flatfile fat #12348
    AnonymousFabian Depry
    Moderator

    Hello Patrick!

    I totally share your excitement about the switch; it might be a painful transition for some, but at the end, it will allow many improvements to the process of transmitting NAACCR data; improvements that wound’t be possible with a limited flat-line data model!

    Your point about missing information in the documentation is well taken, and we are going to see if we can make that clear.

    Technically the trimming and padding rules were only necessary to stay compatible with the flat-file format (although it’s not completely true for the zero-padding rules). But since some software are currently relying on that information, it is unlikely those will be completely removed in the near future. They might be phased out during a longer period of time.

    in reply to: Discussion: The case for preserving naaccrNum #12152
    AnonymousFabian Depry
    Moderator

    To the best of my knowledge, there is no plan (short term or long term) to eliminate the NAACCR numbers from the NAACCR XML specifications.

    It is true that the numbers are optional in the data files, but they are required in the dictionaries; meaning it is not possible to define a data item without defining a unique number for it.

    The reasoning for introducing unique NAACCR XML IDs was that they are human readable, they introduce less potential conflicts (if an organization adds a proper prefix to its own data item, it basically eliminates any possible conflict with other data item IDs), and they are valid programming variables (for most languages), allowing some developers to use them as-is in their programs (it wouldn’t make sense to call a variable 400 but it can be called primarySite).

    The reasoning for making the numbers optional in the data file was that the IDs uniquely identify the data items, and so technically a software shouldn’t need them to consume a data file (if a software prefers to deal with the numbers, then the dictionary provides a one-to-one mapping between the IDs and the numbers).

    At the end, some people/organizations prefer to deal with numbers the way they always have, and some people/organizations have embraced the new IDs, and stopped using the numbers completely. Both approaches are perfectly fine.

    in reply to: NAACCR Fixed-Width data exchange format going away in 2020 #12149
    AnonymousFabian Depry
    Moderator

    Note that the timeline was pushed from 2020 to 2021 to give more time to vendors and registries.

    NAACCR did survey the readiness of the registry recently and provided the results in a ListServ email; here is the link to the archive (the survey was about 2018 readiness, but there were a few XML questions at the end):
    https://share.naaccr.org/community-home/digestviewer/viewthread?GroupId=25&MessageKey=da4e945d-db6d-47c0-afea-514cef5e11fc&CommunityKey=b29e2bb2-73c8-49d4-ad8e-7eb9abcda56d&tab=digestviewer&ReturnUrl=%2fcommunity-home%2fdigestviewer%3fCommunityKey%3db29e2bb2-73c8-49d4-ad8e-7eb9abcda56d

    There was also another ListServ email sent recently to remind everyone of the available resources for the transition:
    https://share.naaccr.org/community-home/digestviewer/viewthread?GroupId=25&MessageKey=91affbfe-5613-4a32-aec3-1f66acbd2522&CommunityKey=b29e2bb2-73c8-49d4-ad8e-7eb9abcda56d&tab=digestviewer&ReturnUrl=%2fcommunity-home%2fdigestviewer%3fCommunityKey%3db29e2bb2-73c8-49d4-ad8e-7eb9abcda56d

    in reply to: Example parsing of a patient record #12148
    AnonymousFabian Depry
    Moderator

    Hello,

    There is no javadoc available for the Java NAACCR XML library, but most of the public methods in that project do have good comments.

    The best example to use the PatientXmlReader is its unit test:
    https://github.com/imsweb/naaccr-xml/blob/master/src/test/java/com/imsweb/naaccrxml/PatientXmlReaderTest.java

    The main class (NaaccrXmlUtils) also has methods that translate flat to XML and XML to flat; those are also good example of using the PatientXmlReader to read all the available fields in a data file (See NaaccrXmlUtils.xmlToFlat() for example):

    https://github.com/imsweb/naaccr-xml/blob/master/src/main/java/com/imsweb/naaccrxml/NaaccrXmlUtils.java

    I hope that helps.

    in reply to: NAACCR Record Version and XML #12147
    AnonymousFabian Depry
    Moderator

    Hello Kathleen,

    We are talking about two different version here:

    1. The NAACCR XML Specifications version
    2. The NAACCR Layout version

    The specifications version (currently 1.4) is tied to the syntax of the XML (when a tag or attribute is removed/added, or when the specifications change, like the maximum number of characters allowed for the NAACCR XML IDs).

    The layout version (currently 180) is tied to the data items (that’s the one the community is used to).

    Both those versions can be updated independently of the other one.

    In 2021, NAACCR will release a new layout (only describing XML since fixed-columns will be retired) and that layout will be version 210 (or v21). There will be a new base dictionary associated to that version.

    At that point, either the specification version will still be at 1.4, or maybe it will have been upgraded to address minor changes to the specifications.

    I hope this answers your question.

    in reply to: Dates in XML #10642
    AnonymousFabian Depry
    Moderator

    That’s correct, the “date” type is the same one that is currently used in the fixed-column files (without the trailing spaces). It’s not related to those more complex “XML” date types.

    in reply to: Sample XML Files #10639
    AnonymousFabian Depry
    Moderator

    Hello Kathleen,

    The samples will be updated to NAACCR 18, but like you said, they are rudimentary; their purpose is to provide a set of “valid” vs “invalid” files that can be used to test a given software.

    There are tools that can create more complete “fake” data files (I know the SEER Data Viewer (https://seer.cancer.gov/tools/dataviewer/) does that and I think there might be others out there, maybe someone else will comment on this); but as far as I know, those tools only set a small subset of values.

    I think a “full” sample file (meaning all variables filled in) would probably need to be crafted by hand. I will bring this topic to the NAACCR XML workgroup during our next meeting.

    in reply to: Dates in XML #10638
    AnonymousFabian Depry
    Moderator

    Hello Kathleen,

    The “date” format is defined in the XML Implementation Guide that is posted on the NAACCR website; here is what those specifications say about that format:

    “A NAACCR-style full or partial date (yyyy, yyyymm or yyyymmdd).”

    And here is the regular expression the specifications define for the type:
    ^(18|19|20)\d\d((0[1-9]|1[012])(0[1-9]|[12]\d|3[01])?)?$

    And so the following are valid dates (no trailing spaces):
    20190615
    201906
    2019

    The format doesn’t allow a known day and unknown year/month or a known day/month and an unknown year.

    Many invalid dates (future dates, or dates with a day that is too high for a given month) will be deemed “valid” by this definition; I think the idea was to be able to use a simple regular expression to define what is acceptable and not acceptable and to let edits deal with corner cases.

    in reply to: Need to build an interface for XML to SQL Datadata #7484
    AnonymousFabian Depry
    Moderator

    Yup, the “naaccr-dictionary-180.xml” is the best source for mapping NAACCR numbers to NAACCR IDs. At least for standard items. For non-standard items, that mapping should be provide in a “user-defined dictionary” by whoever created the XML data files (the data-generator doesn’t support user-defined dictionaries; I wanted to keep things very simple for now).

    in reply to: Need to build an interface for XML to SQL Datadata #7481
    AnonymousFabian Depry
    Moderator

    The standard itself doesn’t require the “naaccrNum” attribute because technically the “naaccrId” is all that is needed to uniquely identify an item. But a lot of software still deal with the numbers, and so it’s convenient to have them.

    I will update the data generator to allow an option to add the numbers to the created file (https://github.com/imsweb/data-generator/issues/30).

    Ultimately, it’s up to you if you want your new framework to require the numbers on incoming data (which is probably more convenient for you), or if you want to follow the strict standard and deal with not always having those numbers.

    in reply to: Tip #1: Running EDITS efficiently on NAACCR XML structures #7477
    AnonymousFabian Depry
    Moderator

    In my opinion, this makes total sense!

    In theory the same concept can be applied to “NaaccrData” edits vs “Patient” edits where an edit on the registry ID item (for example) would only fail one for an entire data file. But I completely understand why supporting that would be much more difficult and there wouldn’t be much gain anyway (there are so little data items at that root level).

    in reply to: Using SAS with NAACCR XML #7473
    AnonymousFabian Depry
    Moderator

    Hi Valerie,

    I looked more into the issue you described, but I can’t reproduce it.

    I used the following file:
    https://github.com/imsweb/naaccr-xml/blob/master/src/test/resources/data/sas/test2.xml

    I tried to create a file that represents the data you described.

    Could you please try that file yourself when you have some time, and confirm that it’s also working for you. And if it is, can you please compare it with your own file and maybe try to figure out the difference?

    Thank you!!!

Viewing 15 posts - 1 through 15 (of 39 total)

Copyright © 2018 NAACCR, Inc. All Rights Reserved | naaccr-swoosh-only See NAACCR Partners and Sponsors