If you don't understand already, you may want to understand the difference between "believing there is no god" versus "not believing there is a god".

It depends on the type of atheist, however, most simply lack a belief in a god. Many atheists become atheists because of intellectual honesty, and doubt (skepticism), and understand that it's unjustified to assert that there's no god at all anywhere, since we haven't finished exploring the whole of existence yet. Most atheists' position on the existence of god is usually "pending further evidence".

This question arises frequently as a way to cast atheists as hypocrites - that we believe something without evidence, while bashing theists for doing the same.

Humanity has a long history of verifiably inventing new religions and gods. Mormonism, for example, was just established in the 1800s. Christianity is one of many active religions, and within this example, has over 30,000 denominations. Even if one of these Christian variants were correct, that means that the other 29,999 denominations were made up. Humanity has worshiped about a million gods. If only one of those is correct, that means that humanity has a 99.9999% making-up-gods rate.

I would expect some actual decent evidence that the 1-out-of-1000000 was the correct god. Otherwise, it's just special pleading.