This paper is an attempt to clear some philosophical questions about the nature of logic by setting up a mathematical framework. The notion of congruence in logic is defined. A logical structure in which there is no non-trivial congruence relation, like some paraconsistent logics, is called simple. The relations between simplicity, the replacement theorem and algebraization of logic are studied (including MacLane-Curry’s theorem and a discussion about Curry’s algebras). We also examine how these concepts are related to such notions as (...) semantics, truth-functionality and bivalence. We argue that a logic, which is simple, can deserve the name logic and that the opposite view is connected with a reductionist perspective (reduction of logic to algebra). (shrink)