Data Science in practice (real world) diff to DS online (Kaggle etc): what does the real data look like?

Data science in real world differs by varied factors from data science in Kaggle. Factors are :

1. Level of complexity of data: The number of tables used for decision-making in an organization is far far higher than used for machine learning in Kaggle.

2. Lack of clear objective in the real world: In Kaggle competition, the programmer knows what to do and how to do. In the real world, most of the time is spent in data analysis and cleansing. This helps us in developing the goal, that can be achieved in the stipulated time.

3. The sparsity of forums for help: Real world data problem cannot be solved by simply looking over the internet. They require careful though process over weeks so that you can develop a plausible strategy.

4. Expectation: Kaggle competition, lay their expectation clearly and it is calculated based on strength of the people participating. In Real world, the expectation of the organization changes every day, week or month, and requires the data scientist to adapt to new expectation. This is essential for keeping the organization competitive.

5. Pipeline formation and descriptive statistics: There is no component of pipeline and descriptive statistics in Kaggle dataset. This is because generally, the problem mentioned in Kaggle is fairly comprehensible where as such is not the case in real world. Data science in the real world requires descriptive statistics for reaching out to non-technical organizational population and pipeline for techies.

Comments

Post a Comment

Popular posts from this blog

13-Weeks of Data Science for Social Good with University of Chicago

Data Science for Social Good (DSSG) is a paramount fellowship program helping data science enthusiast to develop interdisciplinary skills that aim at solving social good problems. DSSG fellowship from the University of Chicago has been involving students, mentors and project managers since 2013 with a motive to provide meaning to data generated in NGOs' (e.g. Ushahidi) and Healthcare organization (e.g. Jose De Mello Saude, Lisbon Portugal). As a selected DSSG fellow in 2017, I seek to apply my machine learning understanding to solve the problem in healthcare in Portugal along with a hybrid team of statistician and software engineers. On the contrary, I was able to build corporate data science skills, teamwork, and social skills which are even more important for the success of the project. 13 weeks time bound workplace of high communication, training and target oriented is summarized sense of DSSG. Before selection, I envisioned DSSG as a program where students (undergraduate or g
Read more

PURPOSE FOR JOINING SOFTWARE DEVELOPERS NETWORK AT TOPTAL

Working smart is designed to take you from dreams distillation to an effective action plan that propels you towards meaningful goals -- Charlene Fike The future that I envisage for myself is that of being a lifelong scholar and developer carrying out path breaking research in the field of Machine Learning and Data Mining and sharing this knowledge with young curious minds through teaching. To realize this career goal and attain excellence in this field through continuous research and implementation of theoretical concepts in real life situations, I covet to join the community of software developer on Toptal. My primary interest focuses in applying data mining particularly graph mining in community detection in brain networks. I worked on a project that involves detection of communities in time across Normal and Alzheimer patients. I have strong interest in data mining, machine learning, computational intelligence and software engineering. I holds a BS degree with first cl
Read more