In this paper we study two lattices of significant particular closure systems on a finite set, namely the union stable closure systems and the convex geometries. Using the notion of (admissible) quasi-closed set and of (deletable) closed set, we determine the covering relation ≺ of these lattices and the changes induced, for instance, on the irreducible elements when one goes from C to C' where C and C' are two such closure systems satisfying C ≺ C'. We also do a systematic study of these lattices of closure systems, characterizing for instance their join-irreducible and their meet-irreducible elements.