Rich Pinder

Rich Pinder

Forum Replies Created

Viewing 3 posts - 1 through 3 (of 3 total)
  • Author
    Posts
  • in reply to: Need to build an interface for XML to SQL Datadata #7472
    Rich Pinder
    Moderator

    Hi all.
    Catching up a bit on last few posts, Jeff you’ve been busy!

    Having realistic and cross’variable logical data to process in useful testing is important for sure, and something another NAACCR WG has been working on for a while. Fabian/Isaac used an initial dataset from this group in the June HackAThon – work does continue in that group. I’ll check to see the status and get back to all.

    You mention that you’re next steps are looking to test the I record ? Looking forward to DB loader tool needs for XML, I think some will want to have the ability to include all record types. Will the tools be able to handle A records??

    Also..in your C# program, are you unpacking the XML to a SINGLE file (usin the KB delim!) … or into SEPERATE files, based on the relations? I was envisioning this temporary/intermediary approach (xml->flat->sqlinsert) might grow more complex as a future XML spec from NAACCR might (will?) move past consolidated data, and into fully nested processing data structures (pat->tum->admission, pat->genetic tests, etc). If that move does happen, would a single file still allow for efficient bulk loads ??

    aok… this is really encouraging to see, and a great help to the community!
    take care
    r

      Identify/create better testing data. The Current XML sample file is a NAACCR version 140 ‘A’ record type. I need to test 160 and 180 versions for record type I.

      in reply to: Need to build an interface for XML to SQL Datadata #7284
      Rich Pinder
      Moderator

      …butt out… oh no! I’d suggest a ‘retired programmer’ is just what’s needed here !!! (how do I insert a grinning emoje here !)

      This approach that you’re all talking about (i think) parallels what some of us talked about long back in the WG history, where it became apparent that the most sexy way to handle schmoozin the XML structured data into relational model would be via an intermediate step where you essentially build relations to emulate whatever structure our NAACCR XML flavor goes to (we have record/pt/tumor now…but down the line… LOTS more comin, me thinks). I think this philosophy of processing our stuff is along same lines you did Fabian for getting XML into SAS (datasets).

      Using the native SQL flavor that a vendor has installed makes a lot of sense. Extra/temp tables also makes sense, where review and manipulation of the data can be done before loading into the production tables.

      Using an open source, file based solution (ie SQLite) would also be a way to offer an alternative to unpack the XML into a ANSI compliant SQL based datastructure, without using a full fledged RDBMS.

      Great THREAD’n here folks… Fabian, mebe this little bunch should petition NAACCR to spin up another of those InPerson ‘workfest’ trips, like we did to initially grok out the XML structure! where did we do that …. was that Atlanta ?? (bad memory chips in this ‘probably should be retired’ programmer) [insert another emoji here, with fingers in ears, wavin hands!]

      adios for now …
      r

      in reply to: Need to build an interface for XML to SQL Datadata #7247
      Rich Pinder
      Moderator

      Thanks… Jeff this a long needed item, and one the community will be grateful for you to work on !! Rich Pinder here, programmer in earlier life and a long time NAACCR member.

      No matter which tools/platforms/approach you take, your work will help for sure! And as a resource for the Vendor community, I’m sure many will be eager to learn from your success.

      An open/generic approach which might work on a number of platforms WOULD be optimum. SQLServer and MySQL come to mind as ones more commonly seen these days by folks who might be interested in loading into SQL. Oracle was our platform of choice long ago, before it became too pricey for us.

      One thought be optimize a C# tool to unpack the XML (using XMLSpy to get the XML structure?), via a Class Library which might be up’gradable as the next Version of our XML def comes into play – but then perhaps look to an open source DB tool, maybe using the slow (but workable) ODBC generic driver approach, to spit out the insert code – that way it may be usable by more systems. Just a thought.

      While back I believe Isaac/Kentucky built something to load directly into relation database (MySQL perhaps?)… perhaps reviewing their approach would be good.

      Thats it for a quick comment – would love to chat (or e’chat) more on the thread as it unfolds. I’m always reachable @ 323) 442-1640

      Rich Pinder
      USC School of Medicine

    Viewing 3 posts - 1 through 3 (of 3 total)

    Copyright © 2018 NAACCR, Inc. All Rights Reserved | naaccr-swoosh-only See NAACCR Partners and Sponsors