Discovering new mathematics might be a process that starts with real things we can see and touch, like: numbers of things, shapes, physics, etc. We then start to notice patterns, things that can be applied in many subjects, this is very useful because we then only have to learn one set of rules which we are able to use in many different situations.
Mathematics differs from other subjects, like physics, in that the criteria for a successful theory is not usefulness or applicability to the real world but only constancy. Uses can follow, but good theories don't have to be useful. This page compares mathematical and physics approaches
As an example of abstraction, think of the concept of number, we can then extend it through integers to the reals and so on. Soon we have complex numbers, matrices, etc.
Finding such underlying simplicity and order has a kind of beauty to us. Our brains seem to be designed to search for patterns, we start to expect to find deeper layers of abstraction. How far can we take this abstraction process? Can we fine one system that underlies everything? A sort of ultimate abstraction, as far as we can get from reality, a sort of pure thought? This is not what appears to happen, it seems that the more abstract we get, we still seem to have a lot of structure. For example, group theory, there seems to be patterns and exceptions to the groups in different dimensions. In some ways it does not seem to be any more structured than subjects that are purely constructs of the human mind like; spelling in the English language, art or music. They all seem to have the level of complexity that the human mind can take rather than being driven by the subject. It seems a bit like fractals, however much we zoom in or out we still have the same level of complexity.
Going back to a slightly more tangible level I still get surprised when mathematical topics that are introduced in certain ways turn out to be connected in unexpected ways, for me examples of this are;
Euler's equation [exp(i*w) = cos(w) + i*sin(w)]
The sandwich form for transforming points using quaternions or multivectors.
Although these things can be easily proved I still am impressed by the connection between things I previously thought unconnected, this sort of structure has a sort of beauty but it also leaves me with an uneasy feeling that the connection is due to some underlying structure that I'm failing to understand. Is there really a deeper structure here or is there such a thing as coincidence in mathematics?
I get the impression that some results in mathematics lead one to believe that there are limits to the level of abstraction that we can archive, for example, the incompleteness theorem.
So given mathematics prime requirement to be self consistent is mathematics truly free of inconstancies? The concept of infinity has always been a problem for mathematics, given its inherent contradictions. We have to be very careful if there is any possibility of division by zero for example and given the reliance of subjects like calculus on the concept we have to be very careful. Mathematicians have built barriers around the problems caused by infinity but I don't know if they have totally eliminated all contradictions from the subject.