A Few Words about Computational Linguistics – By Siddharth

Today was the LingCon Computational Linguistics event, hosted by 42 US, and conducted by Hack+. I went into this event expecting to get a very basic understanding into the field of machine learning and computational linguistics, but what I received was quite different. More info can be found here.

What is computational linguistics? Why is it relevant? What does it do? How does it impact the common person? These are all questions that I had in my head as I went into this conference today, and needless to say, I got most of them answered. Computational Linguistics is the field of rule-based modeling of language, mostly used through natural language processing. We see many examples of it today, such as self-driving cars. Self-driving cars and other processing frameworks like, for example, a voice bot(Alexa, Siri, etc.), all rely on character, speech, or image recognition to work effectively. However, this is no easy task. Think about the number of data points needed to classify a person’s face, for example. You will need to identify the eyes, nose, ears, etc. , and even then, the very task of identifying those said features aren’t particularly obvious. Then how are we, as humans able to do it?

And that right there is the essential question that defines machine learning and neural networking. There must be a way to emulate the human logic into a trainable and teachable program that can identify different features for us. This is exactly what a neural network does, use “neurons” and “layers” to process the data that we give it. Let us refer to the main topic now, language. Let’s say we need to classify a data set of words into positive and negative sets, such that the positive set has a “good” meaning and the negative set has a “bad” meaning. How would we do this?

Well, we can harness the power of tensor rendering, where we turn the training data into a set of matrices and vectors, and apply a simple sigmoid transformation to the points. This will give us a set of clearly defined points that we can use to train the neural network. Now, what we can program the network to do is find a line, with the “slope” as a transformation, such that everything above the line is of one set, and everything below is of another. Then, the network could simply classify all above as say a “1”, and all below as a “0”, for example. From then on, the network would just take in a new data point, apply the transformation, and then just check if it is above or below the defined line. In this way, neural networks can be trained with a set of data to quickly and efficiently process large sets of data in similar classification groups.

Now, obviously this is not perfect, as there are cases where outliers, inaccuracies, and even undetected hardware failures can hinder the effectiveness of a program, and this is why this topic is so relevant today. We have the basis for this technology, like as mentioned before, in the form of Siri and Alexa to even face unlock, but it could be better. This technology has the capability to greatly improve the workflow and effectiveness of day-to-day activities. This is where events like LingCon come in.

We started off with an opening ceremony, with all the usual introductions and agenda. But right after that, we jumped right in. The first workshop I attended was Building a Chatbot by Michael Khait, with Luis.Ai and the Microsoft Bot Framework. This was less hands on and more of a proof of concept, where a chatbot was integrated into a common chat app, in this case Slack.

The second workshop that I attended was Deep Learning for NLP by Peng Qi. For those who don’t know, NLP is natural language processing.  NLP is the general field in computer science that deals with the interaction between computers and language. In this workshop, we learned how a neural network works and what it is used for. In the meantime, I actually got one to work! This network uses the MNIST data set (a very popular public test set) and calculates its own accuracy in its prediction.  This dataset includes many handwritten digits, that can be used for network testing. Here are the results…

 

The neural network in this case actually is accurate in its classification about 96% of the time!

In any case this is a subject that I’d like to delve into much more detail later.

The third workshop that I attended was Character Recognition with PyTorch with Abhishek Sharma. PyTorch is a charming little tool that allows for GPU acceleration when dealing with neural networking, much like tensorflow. In this one, we were able to successfully classify different images very accurately in very little time, wwhich I found was absolutely amazing.

For me, this subject fits right in with my specialties in math. As of right now, I am taking Linear Algebra in school, and that allows me to clearly understand the underlying logic behind this. I love to learn new subjects which are not only interesting in their own right, but that I can integrate with my other interests and strengths. When I am able to understand from the ground up how a mechanism works, it allows me to build and expand upon it more efficiently.

Linear Algebra is the branch of mathematics concerning vector spaces and linear mappings between them. In fact, this mapping is the very basic understanding in machine learning, that one set of data can be easily mapped to another. For example, let’s take a chatbot, for example. One way to make a chatbot recognize user input and textual speech is to map , lets say,  “hi!” and transform it to “hello there!”. In this case, if the user were to input “hi!” then the bot would respond with “hello there!”. In such a manner, Linear Algebra is the integration in machine learning that allows the computer to recognize complex data, like a language.

I think this is why Linear Algebra is so helpful to me: it allows me to visualize data in mass. Maybe not process, but at least visualize. Learning linear algebra forces one to change their perception of how data is manipulated and formatted. Of course, at its bare essentials, Linear Algebra can be stripped down to simple arithmetic, but doing this in the mass is so helpful. For example, mapping each data point one at a time seems so pointless when we can just concadinate them into vectors and matrices and ask a computer to do them as a whole. This is the power of Linear Algebra.

So my concluding thoughts then: I think that although this technology is nowhere near its full capability yet, it is that capability that makes it so interesting. Imagine that a program could go almost level with the human brain on processing tasks. That would have great implications on the future of AI and automation.

I will be attending the full 2-day hackathon session LingHacks in April to continue in this subject.

(For more information on neural networks and machine learning, here is a very good introduction)