Data Science Courses – the missing skills you need

One of the things that I have been complaining about with many of the data science masters courses is that they are missing a lot of the basic skills that are essential for you to be able to be effective in a business situation. It’s one of the things I was going to talk about at the Women in AI event that was postponed this week and I’m more than happy to work with universities who want to help build a course1. That said, some universities are realising this is missing and adding it as optional courses.

Continue reading Data Science Courses – the missing skills you need

A diagnostic tale of docker

twenty sided die showing common excuses for developers not to fix problems, the top of the die shows "Can't reproduce"
Developer d20 gives the answer 🙂 (from Pretend Store)

If you’ve been to any of my technical talks over the past year or so then you’ll know I’m a huge advocate for running AI models as api services within docker containers and using services like cloud formation to give scalability. One of the issues with this is that when you get problems in production they can be difficult to trace. Methodical diagnostics of code rather than data is a skill that is not that common in the AI community and something that comes with experience. Here’s a breakdown of one of these types of problems, the diagnostics to find the cause and the eventual fix, all of which you’re going to need to know if you want to use these types of services.

Read more

Mathematics of player levels in game development

My husband is a game developer and my contributions are usually of the sort where I look at what he’s done and say “hey wouldn’t it be great if it did this”. While these are usually positive ideas, they’re mostly a pain to code in. Today however, I was able to contribute some of my maths knowledge to help balance out one of his games.

Using an open api, he’d written a simple pokemon battle game to be used on twitch by one of our favourite streamers, FederalGhosts, and needed a way of determining player level based on the number of wins, and the number of wins required to reach the next level without recursion. While this post is specifically about the win to level relationship, you could use any progression statistic by applying scaling. Here we want to determine:

  • Number of wins (w) required for a given level (l)
  • The current player level (pl) given a number of wins (pw)
  • Wins remaining to the next level (wr) for a player based on current wins (pw)

Let’s take a look at a few ways of doing this. Each section below has the equations and code examples in python1. Assume all code samples have the following at the top:

import math

database = [
{"name": "player1", "wins": 5},
{"name": "player2", "wins": 15},
{"name": "player3", "wins": 25}
]
Continue reading Mathematics of player levels in game development

Data: access and ethics

Last week I attended two events back to back discussing all things data, but from different angles. The first, Open Data, hosted by the Economist was an event looking at how businesses want to use data and the ethical (legal) means that they can acquire it. The second was a round table discussion of practitioners that I chaired hosted by Ammonite Data, where we mainly focussed on the need for compliance and balancing protection of personal data with the access that our companies need in order to do business effectively.

We’re in a world driven by data. If you don’t have data then you can’t compete. While individuals are getting more protective over their data and understanding its value, businesses are increasingly wanting access to more and more – at what point does legitimate interest or consumer need cross the line?

Continue reading Data: access and ethics

How much maths do you really need for data science?

My LinkedIn news feed was lit up last week by a medium post from Dario Radečić originally posted in December 2019 discussing how much maths is really needed for a job in data science. He starts with berating the answers from the Quora posts by the PhD braniacs who demand you know everything… While the article is fairly light hearted and is probably more an encouragement piece to anyone currently studying or trying to get that first job in data science, I felt that, as someone who hires data scientists1, I could add some substance from the other side.

Continue reading How much maths do you really need for data science?

To everything there is a season

Depiction of Janus, Vatican collection, Photograph by Loudon dodd – Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=7404342

It’s inevitable, at the start of a new year, to reflect on what has gone before and what is yet to come. Janus, the Roman God for whom January is named 1, is depicted in such a way, so it’s difficult at this time of year to be anything other than retrospective :).

Continue reading To everything there is a season

Facebook’s Maths Solving AI

In December, Lample and Charton from Facebook’s Artificial Intelligence Research group published a paper stating that they had created an AI application that outperformed systems such as Matlab and Mathematica when presented with complex equations. Is this a huge leap forward or just an obvious extension of maths solving systems that have been around for years? Let’s take a look.

Continue reading Facebook’s Maths Solving AI

M347 – 33% through and reflections

I’ve recently submitted the first tutor marked assignment of M347, my final course in the BSc Mathematics I’ve been studying with the Open University.  The third unit of this course was long and quite a slog to go through.  While I’ve been using many of these equations over the past few years, diving deep into the theory and derivation has been fascinating, although frustrating due to the lack of practical application.  If you’ve read my other posts then you may recall how frustrated I was with group theory and the early parts of complex analysis, while the quantum world was far more engaging from the start1.  As with all my maths studies, this exercise of filling in the gaps has revealed that there are far more things I didn’t know I didn’t know than things I knew I needed to know.

Continue reading M347 – 33% through and reflections

Let’s talk about testing

One of the things that I find I have to teach data scientists and ML researchers almost universally is understanding how to test their own code. Too often it’s all about testing the results and not enough about the code. I’ve been saying for a while that a lack of proper testing can trip you up and recently we saw a paper that rippled through academia about a “bug” in some code that everyone used…

A Code Glitch May Have Caused Errors In More Than 100 Published Studies

https://www.vice.com/en_us/article/zmjwda/a-code-glitch-may-have-caused-errors-in-more-than-100-published-studies

The short version of this is that back in 2014, a python protocol was released for calculating molecule structure through NMR shifts1 and many other labs have been using this script over the past 5 years.

Continue reading Let’s talk about testing

Getting a first job in AI or data science – what candidates need to know

Me in Lego – well not really, but it does look a lot like me 😉 – this was a very fortuitous collector fig from Series 18.

Getting any role in IT can be daunting as a first timer, whether it’s your first ever job or you’ve changed career or you’ve had a break and are returning as a junior in a new field or anything else.  Getting one in any part of AI can be even more of an up hill struggle.  Job posting and recruitment agencies are asking for PhDs, academic papers and post-doctoral research as well as years of experience in industry.  How can you get past that first barrier?  I get a lot of people asking me this when I present at Meet-Ups so thought I’d collate everything into one post.

I’m going to break down how you can demonstrate the skills that businesses need and how to talk confidently about what you can offer without the fluff.

Continue reading Getting a first job in AI or data science – what candidates need to know