[TODO] visualization from output

[sumanashrestha]

A dashboard which plots schools and center on a map.

• Allow selecting / searching for a SCHOOL and show all the allotted centers. In both map as well as tabular format with some descriptive stats
• Allow selecting / searching for a CENTER and show all the schools whose students are allotted there. In both map as well as tabular format with some descriptive stats

Preferred solution:

• HTML with client side JS which can be hosted statically
• Free and publicly hosted BI

[acharyafutures]

Is HTML and JS mandatory? I am more than happy to help in either of the cases, but was thinking tools like streamlit, solara, etc which works great with data and is convenient!

[samit22]

I can work on it. We can use mapbox. But would require data in the following format.

Consider including input and output inside the sample folder so that it's easier to follow through.

[
    {
        "center": "NATIONAL SCHOOL OF SCIENCES SECONDARY SCHOOL",
        "school": "KATHMANDU MODEL SECONDARY SCHOOL",
        "count": 100,
        "distance_meter": 1330
    },
    {
        "center": "UNIGLOBE MA VI",
        "school": "KATHMANDU MODEL SECONDARY SCHOOL",
        "count": 150,
        "distance_meter": 2300
    },
    {
        "center": "NABODIT BIDHYAKUNJA SECONDARY SCHOOL",
        "school": "UNIGLOBE MA VI",
        "count": 200,
        "distance_meter": 1500
    },
    {
        "center": "PADMAKANYA BIDYASHRAM SECONDARY SCHOOL",
        "school": "NABODIT BIDHYAKUNJA SECONDARY SCHOOL",
        "count": 300,
        "distance_meter": 1700
    }
]

[sumanashrestha]

Is HTML and JS mandatory? I am more than happy to help in either of the cases, but was thinking tools like streamlit, solara, etc which works great with data and is convenient!

Avoiding anything that needs to be self hosted
Any SaaS based solution with reasonable free tier is fine

[sumanashrestha]

@samit22

Consider including input and output inside the sample folder so that it's easier to follow through.

pls run the script with input files in sample directory to generate output file in TSV

one option to read TSV and map to json is d3.tsv

instructions on how to include d3 in browser here, importing only the d3-fetch module should be enough

[ashiishme]

Is HTML and JS mandatory? I am more than happy to help in either of the cases, but was thinking tools like streamlit, solara, etc which works great with data and is convenient!

I can suggest a reliable automated solution:

The visualization can be easily deployed on the GitHub pages with JS & HTML solution and we don't have to worry about any additional stuff.

The GitHub Actions should be able to execute the Python script preparing a JSON file with expected output that is committed back to the repository on the production ready branch.

The production ready branch can be then hosted on the GitHub Pages. Everything can be automated based on the release tag.

[ashiishme]

Is HTML and JS mandatory? I am more than happy to help in either of the cases, but was thinking tools like streamlit, solara, etc which works great with data and is convenient!

I can suggest a reliable automated solution:

The visualization can be easily deployed on the GitHub pages with JS & HTML solution and we don't have to worry about any additional stuff.

The GitHub Actions should be able to execute the Python script preparing a JSON file with expected output that is committed back to the repository on the production ready branch.

The production ready branch can be then hosted on the GitHub Pages. Everything can be automated based on the release tag.

The only downside is, the build time can be high but shouldn't be an issue.

[sebakthapa]

HTML with client side JS which can be hosted statically.

Considering this, I could only think of this idea where each user will have to host their data separately as each user would have distinct input and hence the output.
i.e. they clone the repo, add their data input files, run the script to generate output files, and host it on sth like GitHub pages.

Is this true or am I missing something here?

One way to create a dashboard would be to prompt the user to enter the generated output files and display the file contents but that may not be convenient in case a district or org/someone wants to publish the website to the general public to visualize the exam centers.

[roopeshach]

I think to develop an interactive dashboard we must consider these things, I tried to conceptualize this and here is the summary.
If this features gives a go, I can try working on it.

• Visualizes schools and centers on a map using Leaflet.js.
• Allows users to select a school to display its allocated centers, or select a center to display its corresponding schools.
• Provides descriptive stats such as the total number of students allocated and the average distance to the centers.
• Includes a responsive table layout showing detailed allocation data.
• Employs RESTful API endpoints for efficient data retrieval and management.

This solution will leverage modern web technologies for a seamless and informative user experience. I look forward to any questions or further discussion on this feature's implementation.

[sebakthapa]

@roopeshach Everything seems great but as mentioned static website seems to be the preferred solution, so I think the last point of implementing RESTful API is not applicable.

[RupeshAryal]

Even though it's not a preferred solution as said earlier, I created a streamlit app that fulfills the requirements. Here's my solution:

1. Uses folium to plot centers and schools on openstreetmap tile.
2. Created a new tsv file that contains latitude-longitude data for all the schools(center and non-center).
3. Used pandas to handle all the queries of the available data.
4. Created visualization including maps and tables with streamlit.
5. Hosted the app on streamlit community cloud.

link to visualization : https://centerrandomizedvisualization.streamlit.app/

It works fine for my set of data. However, I still have a lot of refactoring to do for it to be usable for everyone(I rushed it in a way).

[acharyafutures]

Awesome @RupeshAryal , I was also half way on it, maybe we can continue together further? I created somewhat similar!(ss for reference)

[horrormyth]

https://centerrandomizedvisualization.streamlit.app/

Wonderful stuff, I'd love to review this 🎖️ 🎖️

[RupeshAryal]

maybe we can continue together further?

Definitely, I was planning to expand the solution to add few more features that includes users being able to:

1. Filter school based on school address(eg: koteshwor, baneswor, kalanki etc).
2. Filter school according to number of capacity or number of students(eg: <=100, <=500, =>500, =>1000, etc)
3. Generate a output dataset and visualize it by simply clicking on a UI button. Clicking the button will run the center-randomize algorithm generating outputs that will be different from other runs.

1 and 2 will require few data preprocessing and writing queries to retrieve required data.
3 is more complex as it will require making changes to the main algorithm allowing it to output data automatically when it is run.

We can contribute together to build the features, refactor and optimize existing solution and discuss other alternatives as well.

[sumanashrestha]

Even though it's not a preferred solution as said earlier, I created a streamlit app that fulfills the requirements. Here's my solution:

didn't know about streamlit's capabilities. this looks good. please continue

[Devrajkat]

@sumanashrestha, I suggest encapsulating this code within a desktop application to make it more user-friendly for those without a technical background. They could interact with the functionality through a GUI instead of an IDE. Alternatively, if you're considering a web application, I recommend using a framework like Flask or FastAPI. However, keep in mind that this would require a web hosting platform and a database. Please let us know if a desktop app would suffice—I'd be happy to assist with its development.

[dhirajraut1]

@sumanashrestha
#69 Might be able to solve this issue.

[sulochanapradhan]

@sumanashrestha: The long-term and sustainable solution needs to be hosted somewhere. I foresee it as a web application that can be accessed inside the VPC.

[Sandesh4242]

@sulochanapradhan @sumanashrestha
Exploring the possibility of hosting our project infrastructure on a Raspberry Pi presents an intriguing opportunity. Should this idea gain traction, I am fully committed to offering my support in any capacity required. I deeply appreciate your dedication and commendable efforts in advancing our endeavors.