GSoC’17 : Final Submission

Project Title

Database Refactor and Upgrade of LibreHealth EHR

Mentors

Art Eaton and Terry Hill

PR

https://github.com/LibreHealthIO/LibreEHR/pull/599 (Under Review)

For other PRs

https://github.com/LibreHealthIO/LibreEHR/pulls/pri2si17-1997

Issue Opened 

• https://github.com/LibreHealthIO/LibreEHR/issues/647
• https://github.com/LibreHealthIO/LibreEHR/issues/601
• https://github.com/LibreHealthIO/LibreEHR/issues/552

For other issues

https://github.com/LibreHealthIO/LibreEHR/issues/created_by/pri2si17-1997

Work Carried Out 

This project is intended for refactoring of existing database for LibreHealth EHR application. It removes some of the tables which are not used in present application like icd9 codes related tables, documents related tables, etc. It also populates database with random data (as real as possible) which is used for testing database before actual run on real data. It can also be used while adding new feature before implementing that in main code base.

Old Schema v/s New Schema

There were flaws in old schema which is rectified in new one. Major improvements include:

• Lack of Relationship : In old schema, there is no proper relationship between tables and all the linking is maintained in code. In new schema, proper relationship is implemented between tables.

• Inappropriate Field Type : In old schema some fields were inappropriate, like integer is used in place of boolean, varchar is used in place of json type. This is taken care of in new schema.

• Lack of Normalization : Old schema lacked normalization, i.e., tables were not reduced to normal forms. This is taken care of in new schema and every table is broken to its lowest possible form and is properly linked.

• Lack of Comments : This may not be of much use as seen from technical perspective, but having table comment gives proper information of field and their purpose. This has been taken care of in new schema.

EER Diagram of important Modules

• Insurance

• Addresses 

• Users 

• Patients

• Drugs 
• Patient Documents

Seeding and Fake Data Generation

This part includes creation of random data for database. For more details on it, click here.

Future Work

Future work includes :

• Creation of migrations for remaining table after revamping of their UI.
• Creating models and integrate UI with it.

Acknowledgements

I would like to thank my mentors Art Eaton & Terry Hill for letting me do what I wanted and helping me wherever I got stuck, Tony McCormick although he was not my mentor, but he helped in all possible means and that’s the beauty of open source as all people work together and are there to help. Org admins Robert (Robby) O’Connor & Michel Downey for conducting it smoothly and all other members of this organization.

GSoC’17 Week 9 & 10

Hi all, this post is in continuation of my last post. These two weeks were very hectic because of academics, but now everything is on track. In these two weeks I started seeding refactored database schema. Before I proceed further, it is important to know what seeding means? and how it is important for applications?

Seeding is an automated process to initialize database with some fake data, on initial setup of application.

It comes very useful while testing our database queries, before actual run on real data. It also helps in generating test data set, which can be used to test any other feature without affecting real data.

Now comes how to do that? What are challenges in seeding big database? Well the most important thing is to have data with appropriate field type and proper relationship between tables. Data seeded should be as real as possible. It is very difficult to have data according to country, race, some financial, religion specific data, etc. so here it takes to make some manual data, and randomize it. But it still need to be much efficient to have much more diverse and accurate data.

In Laravel we have faker library to seed data (Note that Laravel comes with Faker, but it can also be used standalone. It is not Laravel specific thing). For seeding we make model factories and seeder class. Seeder class calls model factories to seed. Advantage of writing model factories is, we can generate large amount of database records. There is version problem with faker when using it with composer (it can be easily replicated when simply using field as vat, cellNumber, taxId(), etc.). You can also check this thread.

To overcome above mentioned issue, it is necessary to use particular provider.

For eg : $faker_vat = Faker::create(‘fr_BE’);

echo $faker_vat->vat;

For more use case, look at github link (factories and seeds),  and  #647 discussion, for generated sample data.

Till now I have successfully seeded all patient & user related tables and facility table. This PR contains all the codes from refactoring to seeding.

This is it for now. I will update more on this work and GSoC in next post. Stay tuned.

 

GSoC’17 – Week 7 & 8

On July 28, 2017, second evaluation of GSoC ended and I am thankful to my mentors and organization admins for giving me PASS grade. With this second phase of GSoC ended and we are into the third and final phase of Google Summer of Code 17.

In the process of refactoring database, I completed two important tables globals and transactions along with others. The major change in both the tables is introduction of json filed replacing the old generic varchar for storing field’s value.

A simple example from new globals table is :

[
        {
                "tab_name" : 'Appearance',
                "Values" : {
                                "default_first_tab" : 'Dynamic Finder',
                                "navigation_area_width" : 150,
                                .............
                                .............
                        }
        },
        {
                "tab_name" : 'Locale',
                "Values" : {
                                "translate_layouts" : true,
                                "default_language" : 'English(standard)',
                                ...............
                                ...............
                        }
        },
        ............
        ............
]

It was a big change as json field has some limitations as well as some flexibility. Talking about limitations, we can’t use the key as foreign key to link with other tables but it has flexibility to store variable data type (like some field of varchar, some boolean or integers).

It depends on the use case whether to use this field or not. Since in our case, globals table is used to set the configurations of application, json is best suited field. For more insights, look at github issues #552 and #647.  Apart from this I also had a look at logs and ACL (Access Control Lists).

This is for now. I will update more on this in my next post. Stay tuned.

GSoC’17 – Week 5 & 6

In my previous post, I had mentioned about GSoC evaluations and first week of second phase. Now here is the summary of week 2 & 3 of second phase.

After refactoring patient’s and user’s related tables, I refactored ICD-10 codes table (which is a government data).

ICD-10 is 10th revision of International Statistical Classification of Diseases and Related Health Problems . It is a medical classification list by the WHO (World Health Organization) and contains codes for diseases, signs and symptoms, abnormal findings, complaints, social circumstances, and external causes of injury or diseases.

For more information about ICD-10 codes, visit the wiki link.

ICD-9 related tables are purged as they are no longer used. Apart from ICD-10, we have different codes available as CVX , SNOMED, CPT2, CPT4, DSM-IV, HCPCS and RXCUI , intended for different purpose and codes. (There may be some other codes, but I have encountered only these codes.)

After this, I refactored drugs related tables. Inventory modules need to be enabled from Globals. Transaction logs or database logs are very important when you are debugging a big structure, and here also it helped me to figure out table relationship and their execution order. (Making a note of query is also important, as it is my next phase of project.)

I have opened a PR #599 for database refactor, and continuous changes are pushed to my branch.

This is summary of my work till now. For more updates, stay tuned.

GSoC’17 First Evaluation

Evaluations are very necessary to measure one’s performance and progress, and it plays a vital role when you are working remotely, so that your work/progress can easily be monitored. Same is the case with Google Summer of Code. There are three evaluations in this program, where first evaluation was scheduled from June 26, 2017 to June 30, 2017. This is mandatory evaluation for both mentors and students.

Student submit the evaluation of their mentor/s and mentors submit the evaluation of their student/s. For student to remain in program and enjoy the open source culture, he/she must pass every evaluation. Failing in any of the three program means you are no longer a part of this prestigious program.

It always gives you pleasure when you clear your evaluations in any field, not because you get perks, but also it motivates you for future endeavours. I was fortunate enough to receive PASS grade from my mentors Art Eaton and Terry Hill. (Note that if you have more than one mentor, then any one of them will submit your evaluations, and students submit the combined evaluation of their mentors).

After clearing first evaluation, second work period starts and you continue with your work and so did I. After completing most of the core tables, I started refactoring other tables, and tracking query execution.

This is all for now. For further update of work, stay tuned.

Week 4 (June 18 – June 25, 2017)

One more week passed and we reached more close to our first evaluation (June 26, 2017 – June 30, 2017). This post is in continuation to my previous posts and here I will let you know my work progress in this week and some instructions to avoid issues which I faced.

Work :

As mentioned earlier, my initial task is to refactor the database and so, I need to create migrations for all the tables (dropping necessary tables which are not in use, and columns) .  In the last post I mentioned that I had completely refactored the patient and users table, and working on the forms. Now all forms are complete and they have their respective migration file, with some additions and removal of columns from the existing structure. Each form table is related with patient, encounter and users table. Also it is necessary to have your work documented, so added documentation for it along with directory structure.

My next target is to complete the table structure soon and move towards API part.

General Instructions :

Whenever we create something, we face some issues and overcome them, and same is the case here. Few points to remember here is :

• Laravel assumes database to be created in your respective database-server (here it is MySql) and then creates connection to it. So if you want to have artisan command  to create your database, then you need to create one. To create command :

1. Make command using artisan. (php artisan make:command <command_name>)
2. A <command_name>.php file is created in App\Console\Commands directory.
3. Include command signature and business logic for that command. (You can also include arguments.)
4. Register your command in Kernel.php located in App\Console directory, inside $commands.

For more information about creating commands follow the official docs. To see the code look at this link.

• It is very necessary to have the migration created in order i.e, create the base table first and then its corresponding related tables. If we fail to do so, then we can have troubles while creating relationships. It is so because each migration file is prefixed with its time stamp, and they are created in same order when we execute php artisan migrate  command. One work around is to manually change the time stamp of migration file, but this is not recommended as it defeats the purpose of using migrations.

This is all for now. I will update more about my work in my next post.

 

GSoC’17 Proceedings… Week 2 & 3

Time passes by and we are into the third week of GSoC’17. This post includes my work progress and future plans to achieve the final goal. As I mentioned earlier, my task is to refactor the database and bring in the laravel framework without affecting current functionality.

Progress

Till now I have completely refactored `patients` and `users` table. These two are the main tables of LibreEHR. I am using laravel and created migration files for all of these structures. Here is link of my work till now.

A complete refactor is possible when you know each prospects of present structure. So for this I’m in continuous discussion with my mentors on chat and forums. Here is the link of our discussion till now.

Current Work

Right now I am refactoring all forms related tables. These comprises of more than 25 tables. Once completed, its code will be updated on github. Also I am writing database seeders to seed the database. For this I am using faker to have almost real data with proper relationship maintained. This will help in having a clear understanding how data is maintained. This will also come handy while testing database.

Future Plan

In the previous posts, it is mentioned to have a API to communicate with the existing UI and new database schema. So, to have this functional I must refactor all tables and after completion of it I will begin writing models and related controllers. The main idea is to have communication in the form of JSON request and response, means from the UI give request in form of JSON which will be processed and related response (or direct query result) will be returned in the same format. This will be handled by a helper class and data will be presented to existing layout.

Official Coding @GSoC 2017

As soon as community bonding period ends, begins the much awaited coding. This year officially coding starts from May 30, 2017. This is the implementation time where you convert your project plan in working code. Whatever discussion you had with your mentors and drafted plan, get that implemented into your code.

This does not mean that your communication ends with your mentors or community. You have to communicate with your mentors regularly stating your progress and if any road block then how to solve it? They are surely there for help. Mark my words communication with community and mentors is integral part of GSoC.

I am highly thankful to my mentors Art Eaton and Terry Hill and other community members for letting me do the work in my way, knowing that it is very complicated to integrate Laravel in current project.

What is Laravel and how to install it?

Well as soon as you hear Laravel, you will say it is PHP framework for web artisan, and that clearly defines about it. It is a MVC based PHP framework like others as CodeIgniter, Symphone, etc. (You can google the list.)  For more details follow their official documentation.

My work is to bring the database into it and make a way for the existing UI to effectively communicate with it. So before I get into more details of my work, lets see how to install it.

Note : This is for linux (ubuntu 16.04) based systems. You can look at the documentation for windows and mac. It also assumes that you have other PHP is installed on your system.

• Install Composer, a dependency manager for PHP. (sudo apt-get install composer)
• Install laravel via Composer. ( composer global require “laravel/installer” ). After this you need to export PATH variables in order to get laravel executable on your machine.
• Edit you preferred shell profile. For bash : open .bashrc 
• Add export PATH=”$PATH:$HOME/.config/composer/vendor/bin” at the end of file.
• Finally run source ~/.bashrc
• Now create your new application using laravel new command and enjoy its features.

Week 1 (May 28, 2017 – June 2, 2017)

In this week I first created a sample application to communicate with database using laravel API. I made one dummy table and inserted some records in it. After then I created model and controller for it, and configured its route. Finally making a valid request either through curl or opening a link in browser will give the results in JSON format. Here is the git hub link. Note that it is dummy project. You need to create database and set the password in .env file.

After this I discussed with my mentor and regularly updated him with proposed structure for its database tables and seeking his advice for any further improvement. Here is the link of all discussions on github.

I will update my further work in my next post.

Community Bonding

Well this post is in continuation of my previous post about GSoC. Soon after projects are announced, community bonding period starts. So lets start with it.

What is Community Bonding?

The community bonding period is all about, well, community bonding. Rather than jumping straight into coding, you’ve got some time to learn about your organisation’s processes – release and otherwise – developer interactions, codes of conduct, etc.

This is the time when you get to know about your mentor and dig deeper into code base. You have to get refined your proposal into project plans and how you will do that with approximate deadline.

This year it started from May 4, 2017 (Soon after the announcement of projects) and will end on May 29, 2017.

Week 1 & 2 :

Well I would say that I was not very much active on community forums and IRC chat during this period, because I was having end semester exams. But I was able to find some time to look at code base and get to know its work flow. (Since mine proposal was for database upgrade and framework integration, so I mainly concentrated on that.)

It is advisable to get into touch with mentors, as they know better how to approach for your project? and you will get more inputs.

Week 3 :

This was the week when my exams were over and I can concentrate properly on my project. I started discussion with my mentors and working at database.

Present Work:

Right now I’m making a sample API to communicate with Eloquent ORM (laravel part) and our layout. If this works then there will be no issue with integration. Also I am making note of details of each table (its purpose, to which tables they are linked and details of each field.)

Work to be completed before May 30, 2017 :

My target is to get a clear picture of the present database and discuss all the possibilities of making a good structure. Also I will make my API work (this is a sample API and if it works then I will extend it. All things considered are dummy.)

After completing these two I will make plan for final execution of project along with deadline.

I my next post I will give more detail of my work.

 

Google Summer of Code’17

Hi all, I’m Priyanshu Sinha, and this is my first blog post, which will be followed by many in coming months as I got selected in one of the most prestigious program i.e, Google Summer of Code under the organisation LibreHealth.

Now let’s start with what is GSoC?

Google Summer of Code is a global program focused on introducing students to open source software development. Students work on a 3 month programming project with an open source organization during their break from university.

Since its inception in 2005, the program has brought together 12,000+ student participants and 11,000 mentors from over 127 countries worldwide. Google Summer of Code has produced 30,000,000+ lines of code for 568 open source organizations. 

This year more than 200 organizations got selected. Full list can be viewed at GSoC 2017 organizations. One of these is LibreHealth.

About LibreHealth

LibreHealth is collaborative community for free & open source software projects in Health IT, and is a member project of Software Freedom Conservancy.

There were three projects  under this organization as:

• LibreHealth Toolkit (A foundational base for building Health IT tools.)
• LIbreHealth EHR (An electronic health record derived from best practices and technology from leading open source systems.)
• LibreHealth Radiology (A specialised distribution of LibreHealth Toolkit customised for radiology health care professionals.)

I would be working on LibreHealth EHR. So let’s get into some details of this project and what would be my contribution?

The LibreHealth EHR application is a clinically-focused electronic health record (EHR) system designed to be both easy to use “out of the box” and also customizable for use in a variety of health care settings. It builds on the strength of the LibreHealth Toolkit, and adapts many of the proven user experiences built over many years in the OpenEMR system. It is designed to be a modern, easy-to-use experience for health care professionals in their daily work.

Technology used is PHP(native) with HTML and JavaScript.

During this summer I’m supposed to upgrade the database to PDO(PHP Data Objects) and integrate MVC (Model-View-Controller) framework to the existing project. Presently there is no framework in this project. This aims for a gradual refactoring of existing project without affecting the present working.

Journey to GSoC

After list of organizations were announced on February 27, 2017, I opted for this organization, and made contributions to it. My contributions can be seen here. While making contributions, I was able to clear my git concepts (i.e, how git actually works). Finally the results were announced on May 4, 2017 and I was delighted to see my project in the list.

My Mentors

Art Eaton & Terry Hill. These are my mentors who will guide me during this period.

Community

All the members in this community are very kind and helpful. Despite of the difference in timezone, they helped us with almost no delay. It is really a great experience to work with this community.

At last, I would like to thank all member of this community especially Art Eaton, Tony McCormick, Terry Hill, Robert (Robby) O’Connor, Michel Downey and Judy Gichoya.