Machine learning and materials science

This post is a translation of a post that appeared on my Swedish blog in May 2019.

The other week I read an optimistic blog post on the subject of machine learning by the American skeptic and neurologist Steven Novella. He wrote, among other things, about an American research group that has trained a neural network to determine properties of doped graphene, that is graphene where some of the carbon atoms are replaced with other elements, from the placement of the dopant atoms. Novella chose to portray this as the neural network being able to perform decades of research in the course of a few days, and hinted that this could give us practical applications of graphene considerably earlier than if no machine learning had been used.

As someone who is interested in both graphene and machine learning, I obviously had to find the scientific paper the group had published and try to figure out what they had actually done.

The research question
The paper in question is published in npj Computational Materials (it is also open access, by the way) and according to the title it deals with the prediction of the so-called band gap of materials that are a combination of graphene and boron nitride. Boron nitride is a material that consists of two types of atoms, boron and nitrogen, arranged in a hexagonal lattice just like the carbon atoms in graphene. Also just like graphene, boron nitride can be produced as just a single, super-thin layer of atoms. These similarities between the two materials are a part of the reason why people try to combine them.

Another part of the reason is that while graphene has excellent electrical conductivity, it is very difficult to get boron nitride to conduct electricity at all. This difference is due to that it requires fairly little energy to get the electrons in graphene moving, while the electrons in boron nitride need a lot of extra energy to get to a state where they are mobile. This energy boost that is needed for the electrons to be able to move is also a measure of the band gap (corresponding to a gap in energy between different states that the electrons can be in). Graphene thus has an extremely small band gap, boron nitride has a large bandgap. By combining the two materials people want to create a hybrid material with a band gap of a size that is useful for e.g. applications in electronics.

However, it turns out that you cannot just replace a few carbon atoms with boron and nitrogen. How the boron and nitrogen atoms are arranged in relation to each other matters for how large the band gap of the resulting material turns out to be. What the American research group has done is try to predict the size of the bandgap based on the placement of boron and nitrogen atoms using artificial neural networks, more specifically so-called CNNs or convolutional neural networks.

The neural networks

CNNs are a type of neural networks that have been developed to pick out characteristic features from images and then classify the images based on the features - they are for example useful for facial recognition and when self-driving cars need to tell the difference between a pedestrian and a road sign. The basic principle of a CNN is similar to comparing small regions of a picture with smaller, simpler images and give a positive response if they are similar. If for example you have a picture of a house and the smaller image has a vertical line you might get a positive response when you get to the corners, doors or windows since their depictions contain straight, vertical sections. In  CNN, however, you have to represent both images as matrices of numbers, and you also have several layers where the result of one comparison to a smaller image in turn is compared with more matrices (this is needed for the identification of more complex features in the image). Also note that the smaller image (or filter) is not something you define beforehand, but something the network learns. If your training data contains no straight lines, the filters that result from training probably will not do so either.



To be able to use CNNs for the graphene problem described above the researchers chose to use computer models where each pair of atoms is represented by numbers. When they introduce boron and nitrogen atoms in graphene they usually come in pairs, with a boron and a nitrogen atom next to each other. The researchers therefore chose to represent a boron-nitrogen pair with a "one" and a carbon-carbon pair with a "zero", and thereby constructed an image of the material that different types of CNNs can handle. They also built their networks to give the size of the band gap as output data.

Neural networks need to be trained with relevant data in order to work, something that usually involves automatically comparing the output of hte network to the desired result, calculating the deviation, and adjusting the neural network to give a better answer. In order to train their neural networks the researchers therefore generated several thousand possible configurations and calculated the bandgap of each configuration using density functional theory. The trained networks were then used to predict the bandgap for another batch of configurations where the calculated bandgap was known but that were not used in training. The results turned out to be very promising.

What can we learn from this?

So what is the effect of this study? The researchers have successfully shown that it is possible to predict certain properties of materials using neural networks, which should give those who do research on graphene and other two-dimensional materials another tool that they can use in their research. There is still a long way to go from this particular study to electronics based on graphene and boron nitride, but it may make it easier to know what kind of material configurations are worth working on.

Another interesting thing about this study is what it says between the lines about the limits of machine learning. For the method to work at all the neural network needs to receive all the relevant information in a format it can process, which means that it requires quite a bit of knowledge about graphene and boron nitride to even formulate the problem in such a way that it can be tackled. For example, in this study the researchers have chosen to focus completely on where each boron-nitrogen pair is situated in relation to other pairs and thus discarded all other characteristics of the material, presumably based on what is already known about these materials. (As a example, the relative orientation of neighbouring boron-nitrogen pairs is completely ignored - is it boron-nitrogen-nitrogen-boron or boron-nitrogen-boron-nitrogen? This information has been trimmed away before the neural network is involved.)

A known limitation of neural network is that it is hard to understand why they work the way they do even when they give good results. In a study like this one it would have been very interesting to see what the structures with low or high bandgap respectively have in common, but that is not information that is easy to extract from the neural network itself and the researchers seem not to have made any effort to try. I strongly suspect that a method to understand what is going on inside the networks is necessary for this type of study to be helpful in understanding the studied materials.

As you have probably understood by now, I do not quite agree with Steven Novella about this one fairly limited study showing that neural networks will do decades of research in a few days and take us significantly closer to graphene electronics, but the results in it are still interesting as an example of machine learning in materials physics.

Why do we not have invisibility cloaks?

This post is a translation of a post that was published on my Swedish-language blog in December 2018.

In 2018, it was ten years since I stumbled upon the opportunity to write my Bachelor's thesis on the subject of photonic crystals. Photonic crystals are a metamaterial, an artificial material created by putting together small pieces of ordinary materials to get something with unusual properties. A photonic crystal can for example efficiently block electromagnetic radiation (light, microwaves etc.) in a specific frequency band, while frequencies outside the band get through with relatively little loss. Materials with this property could have many interesting applications, but the application that caught the attention of popular science publications at the time was that metamaterials could maybe, maybe be used to make things invisible. The obvious question is then, have we gotten any closer to having invisibility cloaks over the last 10 years?


Ten years ago it was photonic crystals specifically that people were talking about. A photonic crystal consists of at least two materials that have different optical properties - usually materials that are not electrically conducting and that have different refractive indices. It could for example be two different types of plastic or plastic and glass. The two materials are placed in a periodic structure, which for example could mean a stack of thin slabs where every other is plastic and every other is glass (Wikipedia has some excellent figures that shows what this can look like). The thickness of the slabs should correspond roughly to about half the wavelength of the electromagnetic radiation that you want to stop from propagating in the material.

Just stopping radiation of a specific frequency from propagating is, however, not enough to make something invisible. Preferably, we would like to lead the light around the object that we want to render invisible, so that the eye of people looking at it only receive light from whatever is behind the object. Theoretically this is possible with photonic crystals, due to something called effective negative refractive index.

When light passes from one material to the other, for example from air to glass, it does not continue in a straight line but changes direction with a specific angle. The angle depends on the difference in speed of light between the two materials, and this difference is expressed in terms of the refractive index. In normal materials the refractive index is positive, but in metamaterials radiation at some frequencies can change direction with a much larger angle than is ever possible in a normal material (see figure). This is expressed as the metamaterial having an effective negative refractive index. By adjusting the metamaterial to get a specific value of the refractive index it is possible to use this to control the path of light, for example to lead it around an object you want to conceal.

However, there are two problems with this. Firstly, the frequecy range where photonic crystals have a negative refractive index is usually very narrow, so a given metamaterial will only work for a small number of frequencies (say for example you would be invisible in green light but not in red or blue). Secondly, even if every layer in the photonic crystal is very thin you need a lot of layers to get a good effect. This means that even for visible light, with wavelengths below one micron, an invisibility cloak based on photonic crystals would be quite unwieldy.

Since my first contact with this field, another type of metamaterial has become more popular. Instead of mostly using non-conducting materials, the materials are built up from metallic elements or even small electric circuits. This category of metamaterials mostly builds on various resonance phenomena that can occur in metallic materials that are exposed to electromagnetic radiation. A common example is so-called split-ring resonators, that consist of to rings of metallic material where one is smaller and placed inside the other one. The rings both have an opening and they are placed so that the openings are opposite of each other. When this structure is exposed to electromagnetic radiation, this causes an electric current in the rings (induction) and electromagnetic charge will build up around the openings (capacitance). These currents and charges in turn affect surrounding electric and magnetic fields, i.e. the radiation.

A difference between traditional photonic crystals and metamaterials with metallic components is that with the metallic components the individual elements in the metamaterial, like the ring resonators for example, should be much smaller than the wavelength of the radiation that is to be stopped or controlled. This is a good thing if you want to make an "invisibility cloak" for lower frequencies (i.e. where the the wavelengths are larger) but for visible light you might run into the problem that it is still fairly difficult to make large quantities of components that are just a few tens of nanometres in size, especially since they need to be manufactured with high precision. In addition, the metallic metamaterials also only work in a limited frequency band that depends on the size of the elements. The frequency band is a bit wider than for photonic crystals, but from what I have been able to find it would still be difficult to cover for example the entire visible range. Thus, no invisibility cloak à la Harry Potter yet.

As a final note, I should probably mention that even if an invisibility cloak for the visible spectrum would be cool, that is not really what drives research in this area. Most papers I have found deal with electromagnetic radiation with wavelengths from a few millimetres up to several centimetres, which are used for e.g. radar and communication. In this frequency range metallic metamaterials work fairly well and have given applications for those who for example want to make airplanes harder to detect with radar.

About radars and 'the spinning thing'

This post is a translation of a post previously published on my Swedish-language blog in September 2018.

Something I have noticed during the last few years is that if someone asks what I work with and I answer 'radar development', there is a very common follow-up question. Namely, 'does it have one of those spinning things?'

Usually, 'spinning thing' here means the typical radar display that you see in ships and airplanes, especially in movies and tv-series, with a greenish beam that sweeps around a circular display showing bright dots where the radar sees a target. Of course, sometimes 'spinning thing' also refers to the antenna in itself, since ship radar antennas are usually mounted fairly high and keep spinning around. It's a fairly natural question, since these types of radars can be found in popular culture and are in any case very visible.

As it happens, though, the types of radars that I work with do not have any sort of spinning thing. If I say so, the next question is usually something along the lines of 'then how does it work?' which is complicated and takes time to explain. It's also a very good question, so I thought I would take the time to explain it here.

If you think a bit more about that classical green radar display you quickly realize that it is a very good way to show two things: Distance to the thing the radar is detecting, and the direction the antenna was pointing in when the object was detected. Distance is one thing that is very straightforward to measure with radars: You send out a signal or radio wave pulse and wait to see if it is reflected by anything and therefore comes back, and then you calculate the distance from the time elapsed between emission of the pulse and reception of the reflected pulse. Direction is often more difficult, but for the spinning ship radar there is a simple solution - the thing the radar sees must be in the same direction as the antenna is facing when the reflected signal was received. 'The spinning thing' is thus something that helps you determine in which direction the radar target is located.

If your radar does not have a spinning thing, you need to find another way to determine which direction a signal or pulse of radio waves comes from. One way to understand how this can be achieved is to start with a type of waves that are easier to visualize, say water waves. If you are angling at a lake, your float (the red thing in the figure below is supposed to be the float, by the way) will bob up and down with the waves, sometimes being at the top of a wave and sometimes in the trough between waves.

Now imagine that you instead, for some reason, have four floats in a row in the water. If the waves are coming straight towards you, all your floats will be on the peak of a wave or in a trough at the same time. If the waves are coming from some other direction, some of your floats could be in a trough while others are not. Even if all four are affected by the same wave motion, the phase of the wave can be different for each float. If you know the wavelength and speed of the waves and the distance between the floats, you can calculate the (approximate) direction that the wave comes from the vertical position of the floats.

You can also imagine that you, on a day when the lake is calm, equip your floats with something that can create waves instead. The waves created by the different floats will interfere with each other, so if two peaks from two different floats end up in the same place at the same time they will reinforce each other, but if a peak and a trough end up in the same place at the same time the waves will cancel each other out. The end result of this will be that the resulting waves are steered in a specific direction depending on the relative phase at the different floats.

The connection between this thought experiment and radars is that the antenna elements in a radar that does not have 'the spinning thing' (say a front-looking car radar) act a lot like the floats. If the antenna that your radar emits radiation from has multiple antenna elements, you can decide that they should transmit the radio waves with small differences in phase. The emitted waves will cancel each other in some directions but not in others, with the effect of steering the resulting wave in a specific direction. By changing the relative phase between the elements it would be possible to change the direction and thereby scan the surroundings.

Another common alternative is if the receive antenna has multiple antenna elements. In this case you could have a somewhat simpler transmit antenna that illuminates a larger area with radio waves. When the waves are reflected by objects in this area they are received by the receive antenna, and the relative phase of the signal at the different receive antenna elements gives the direction of the object.

Of course, there are things that make this more complicated in reality - for example, you may need quite a lot of antenna elements if you want to look at a large area with high resolution. It can also be difficult when there are multiple objects that reflect the waves. A thing that makes the last problem easier is that reflections from objects at different distances will reach the radar at different times, so they can be separated with respect to distance before the direction is determined. In automotive radars the radial velocity of the targets relative to the radar is usually also determined, which provides another way of separating targets. An approaching car and a lamppost at the same distance from the radar can be separated based on their velocity, and then you can calculate the direction of each object separately.
Even without 'the spinning thing'.