You have three options here. My approach is to calculate volumes for all of them and see which one is greatest.
1. Cylinder's base rests on a 4x5 side of the box. So, height becomes 6 and the maximum possible radius becomes 4/2.
So, maximum volume = \(\pi . (4/2)^2 . 6 = \pi 24\)
2. Cylinder's base rests on a 5x6 side of the box. So, height becomes 4 and the maximum possible radius becomes 5/2.
So, maximum volume = \(\pi . (5/2)^2 . 4= \pi 25\)
3. Cylinder's base rests on a 4x6 side of the box. So, height becomes 5 and the maximum possible radius becomes 4/2.
So, maximum volume = \(\pi . (4/2)^2 . 5 = \pi 20\)
Option 2 gives maximum value and so that should be the answer,\(\pi 25.\)
In hindsight you can see that since in volume of a cylinder \((\pi r^2 h)\) the radius r has a higher power, the one with a higher radius will have maximum volume when other parameters are close to each other (like 4,5 and 6). You get maximum radius when base is kept on a 6x5 face. Still, personally I will go with finding all three volumes and then comparing them.