If you're building a Windows 98 PC to run software that doesn't work in modern versions of Windows (for example, an old synth editor app), you'll probably want to install the newest Win98-compatible versions of some apps.
Windows 98 USB storage support is not great. You'll need to unstall nsusb33 for things to work properly.
Version 2.0 of the MSI installer redistributable is the latest version for Windows 98. It's required to install some .msi software, such as the Foxit Reader listed below.
The most modern browser available for Windows 98 is Opera 9.64. It was released on March 3, 2009 and is the most advanced and standards-compliant of the browsers available for Win98. It is followed closely by Firefox 18.104.22.168, which was released on December 18, 2008.
The December 2006 version of DirectX is the last version to support Windows 98.
Acrobat Reader 5.05 was released in 2001 and supports Windows 98. Acrobat Reader 6.0 may work on Windows 98 SE, but it was not officially supported. In addition, version 6 was considerably slower than 5. Foxit Reader 2.3 also works on Windows 98.
Winamp was arguably the best music player available for Windows 98. Version 5.34 is the last compatible version.
Winzip 10, released in 2005, is the last version that works properly with Windows 98. It is a shareware program, and it may or may not be possible to buy a license for it now. IZArc 3.6 also works well for file extraction on Win98.
The last version of OpenOffice to work is the 2.4.x series. 2.4.3 is a good choice.
Office XP is the last version of Microsoft Office that runs on Windows 98.
The Yamaha UX16 and UX96 MIDI interfaces work well with Windows 98. Version 1.22 is the last version of the driver for 98. This should also work for some other Yamaha USB MIDI devices such as the UW500 and many of their USB-enabled synthesizers.
Yamedit is a shareware patch editor for the Yamaha YS100, YS200, DX11, DX21, and TX81Z. It was released by Blue Bear Software in 1997 and won't run on modern versions of Windows. It's not possible to register it, so you won't be able to save patches, but its ability to randomize patch parameters is nice.
The newest version of Adobe Photoshop that supports Windows 98 is 7.0.1.
For FTP and SFTP file transfers, WinSCP is one of the best programs. Version 4.07 works on Windows 98.