P%2FData Expertise

Error converting content: marked is not a function

icon:: 🛠

type:: project
status:: active
sort-key:: 2023.10
start:: Oct 6th, 2023
estimated-end:: Dec 31st, 2023 
end:: Jan 19th, 2024 
duration:: 4 months
score:: 🌟🌟🌟🌟🌟
- **Next Action**
- DONE update purpose and outcome for P/Data Expertise
	  :LOGBOOK:
	  CLOCK: [2022-06-27 Mon 10:28:07]
	  :END:
- TODO turn this into a project task
  - Block Reference
- **All TODOs on this page**
	  collapsed:: true
  - {{query (and (task todo) P/Data Expertise )}}
- **Important Dates**
- Oct 6th, 2023: project init
- **Outcome Visioning**
- **Purpose & Outcome**
  - Be expert level comfort in taking a data source in various formats (csv, excel, api, scraping) and be able to analyze it with ease by describing, slicing, dicing, plotting, charting and what not. ✅
  - Explore some awesome glorious datasets. Learn to handle time series data. Train a model or two. Complete some mini projects successfully ✅
  - Python expertise in numpy, panda, matplotlib and pytorch; Read Books/Python Data Analysis ✅
  - Bonus: Revisit Quarto for writeups
- **Wins**
  - PostgreSQL
- **Final 10 Days**
  collapsed:: true
- Re-review purpose and outcome and plan out last 10 days for wrapping; #nice #ClosingTheLoop🧶
	  id:: 6585e172-3bef-4c31-862d-d098ad4386d0
  - "Be expert level comfort in taking a data source in various formats (csv, excel, api, scraping) and be able to analyze it"
  - Ok. I need a generic loader fucntion? Turn the representation to pandas? Hmm.
  - Aite. Excel data loaded. 80MB file. 127,939 rows.  Took 1 minute 10 seconds. That autocomplete from #copilot on LCA dataset though! #datasets
			  collapsed:: true
    - ![image.png](../assets/image_1703278804742_0.png)
    - Ok. rows for 2023 Q4 matches their statistics summary report. 127,939 application processed. Nice. Love when data matches.
				  collapsed:: true
    - ![image.png](../assets/image_1703278984962_0.png)
    - Ok. Very interesting stuff. This is real knowledge. Creating P/LCA Data just capturing knowledge. Bonus - deploy a useful website for exploring this data
    - ![image.png](../assets/image_1703279540230_0.png)
  - "Explore some awesome glorious datasets"
    - Let's hit the API and plot some charts.
    - Data Data Data. Found #H1B Salary data via #US DOL 🍅
    - Ok. #1teer2nishana🎯 - US DOL LCA data. Reports in XLS format so I get to load and read from this format and explore this dataset I always wanted to know about? Thnaks H1BData.info #H1B Salary
    - https://www.dol.gov/agencies/eta/foreign-labor/performance
  - TODO "Learn to handle time series data."
  - TODO "Train a model or two."
  - "Complete some mini projects successfully"
		  collapsed:: true
  - P/Business Management v1 ✅
  - P/Osho AI in progress
  - "Python expertise in numpy, panda, matplotlib and pytorch;"
		  collapsed:: true
  - so far so good; mostly pandas
  - "Revisit Quarto for writeups"
		  collapsed:: true
  - hmm - maybe for next 90-90-1 Project - P/Writing Expertise - which I have been contemplating much this week. #googsegueᴝ
- - Project Summary
  collapsed:: true
- Jan 22nd, 2024 wrap notes
  - A  start project. This has changed me in a way that I can't go back in time. I can do data analysis. Crunch files and data and create charts at ease. In fact, I like it so much. Nothing more needs to be said. Above and Beyond Expectations.
- Started this project on Oct 6th, 2023
- Milestones
  - Nov 17th, 2023
  - {{embed Block Reference}}
- Checks
	  collapsed:: true
  - TODO Project Wrap Up
  - TODO Extract and Create Information Packets
  - TODO Clean up
  - TODO Write Summary
  - TODO Add any relevant tags
- ### Resources
  collapsed:: true
- Week 1 Notes
	  collapsed:: true
  - numpy and panda
  - Key datastructure - Series and DataFrame
  - FastAI C22 Part 1 conitnues with lec 5 - wrangling titanic dataset
- Books
	  id:: 6525a29f-d9eb-490e-8e87-4b9c37a505bf
	  collapsed:: true
  - Books/Python Data Analysis ✅
  - Books/Fundamentals of Data Visualization
  - Books/Fluent Python
  - Books/The Big Book of Dashboards
  - Really Bad Books 🤮
  - Books/How Data Happened ❌
  - Books/Data Science in Context ❌
  - Books/Fundamentals of Data Engineering ❌
- Tech
  - shadcn-ui
  - Tech/TanStack Table
  - Tech/Recharts
  - PostgreSQL.app
  - Really good - https://bitestreams.com/blog/fastapi_template/ & https://bitestreams.com/blog/fastapi_sqlalchemy/
  - Tech/SQLAlchemy
  - [Declarative Mapping](https://docs.sqlalchemy.org/en/14/orm/mapping_styles.html#declarative-mapping)
  - Tech/DuckDB
  - Tech/JupySQL - sql in notebook! no more need to kill self with finding db clients
- TODO Spatial Data Management
  - https://github.com/giswqs/geog-414