You have the scene where Vito is shot, and Fredo just sits on the curb and cries. That establishes that he doesn't have the nerve to function under pressure.

Then, when Vito comes home from the hospital and is recuperating in the bedroom, everyone else leaves, and Fredo comes in and sits in a chair. There's no dialogue in the scene but it is still very powerful. Fredo can hardly look at his father because he knows he has failed him. Vito looks at Fredo and knows he can never play a really useful role in the family. The scene conveys those emotions very well.