LESS COMMON CAUSES OF MEMORY LOSS Other conditions that can lead to problems with memory include: • Infection: Memory loss may be attributed to severe infection around the brain, including neurosyphilis. • Head injury: Symptoms of a mild brain injury may include confusion and trouble with memory and concentration. • Tumors: Memory and the ability to process information may be affected by brain tumors, the American Brain Tumor Association says. In addition, treatments for a tumor can affect your memory, including brain surgery, chemo, or radiation therapy. • Alcoholism, Substance Abuse: Both alcoholism and drug abuse can affect memory, men who drank heavily showed signs of mental decline one to six years earlier than light drinkers.