The intense, sometimes debilitating discomfort you feel in the days after a workout is known as delayed onset muscle soreness (DOMS).
It results from microscopic damage to the muscle fibers, particularly in "untrained" muscles (e.g., when you do new exercises your body is not yet accustomed to.)
Exercises that cause DOMS involve what is known as eccentric muscle contractions—when the muscle is engaged at the same time it is lengthening, such as biceps during the downward motion of a curl.
DOMS can set in anywhere from 24 to 72 hours after exercise and can persist for days—the key word in DOMS being "delayed."
Soreness is a side effect of the repair/recovery process. As you continue to recover, it makes sense that the soreness would subsist.