I understand how you feel. I found function composition to be quite difficult to grasp at first too. What helped me grok the matter were type signatures. Consider:
(*) :: Num x => x -> x -> x
(+) :: Num y => y -> y -> y
(.) :: (b -> c) -> (a -> b) -> a -> c
Now when you write (*) . (+) it is actually the same as (.) (*) (+) (i.e. (*) is the first argument to (.) and (+) is the second argument to (.)):
(.) :: (b -> c) -> (a -> b) -> a -> c
|______| |______|
| |
(*) (+)
Hence the type signature of (*) (i.e. Num x => x -> x -> x) unifies with b -> c:
(*) :: Num x => x -> x -> x -- remember that `x -> x -> x`
| |____| -- is implicitly `x -> (x -> x)`
| |
b -> c
(.) (*) :: (a -> b) -> a -> c
| |
| |‾‾‾‾|
Num x => x x -> x
(.) (*) :: Num x => (a -> x) -> a -> x -> x
Hence the type signature of (+) (i.e. Num y => y -> y -> y) unifies with Num x => a -> x:
(+) :: Num y => y -> y -> y -- remember that `y -> y -> y`
| |____| -- is implicitly `y -> (y -> y)`
| |
Num x => a -> x
(.) (*) (+) :: Num x => a -> x -> x
| | |
| |‾‾‾‾| |‾‾‾‾|
Num y => y y -> y y -> y
(.) (*) (+) :: (Num (y -> y), Num y) => y -> (y -> y) -> y -> y
I hope that clarifies where the Num (y -> y) and Num y come from. You are left with a very weird function of the type (Num (y -> y), Num y) => y -> (y -> y) -> y -> y.
What makes it so weird is that it expects both y and y -> y to be instances of Num. It’s understandable that y should be an instance of Num, but how y -> y? Making y -> y an instance of Num seems illogical. That can’t be correct.
However, it makes sense when you look at what function composition actually does:
( f . g ) = \z -> f ( g z)
((*) . (+)) = \z -> (*) ((+) z)
So you have a function \z -> (*) ((+) z). Hence z must clearly be an instance of Num because (+) is applied to it. Thus the type of \z -> (*) ((+) z) is Num t => t -> ... where ... is the type of (*) ((+) z), which we will find out in a moment.
Therefore ((+) z) is of the type Num t => t -> t because it requires one more number. However, before it is applied to another number, (*) is applied to it.
Hence (*) expects ((+) z) to be an instance of Num, which is why t -> t is expected to be an instance of Num. Thus the ... is replaced by (t -> t) -> t -> t and the constraint Num (t -> t) is added, resulting in the type (Num (t -> t), Num t) => t -> (t -> t) -> t -> t.
The way you really want to combine (*) and (+) is using (.:):
(.:) :: (c -> d) -> (a -> b -> c) -> a -> b -> d
f .: g = \x y -> f (g x y)
Hence (*) .: (+) is the same as \x y -> (*) ((+) x y). Now two arguments are given to (+) ensuring that ((+) x y) is indeed just Num t => t and not Num t => t -> t.
Hence ((*) .: (+)) 2 3 5 is (*) ((+) 2 3) 5 which is (*) 5 5 which is 25, which I believe is what you want.
Note that f .: g can also be written as (f .) . g, and (.:) can also be defined as (.:) = (.) . (.). You can read more about it here:
What does (f .) . g mean in Haskell?