We introduce the concept of a geometric categorical $\mathfrak{sl}_2$ action and relate it to that of a strong categorical $\mathfrak{sl}_2$ action. The latter is a special kind of $2$ -representation in the sense of Lauda and Rouquier. The main result is that a geometric categorical $\mathfrak{sl}_2$ action induces a strong categorical $\mathfrak{sl}_2$ action. This allows one to apply the theory of strong $\mathfrak{sl}_2$ actions to various geometric situations. Our main example is the construction of a geometric categorical $\mathfrak{sl}_2$ action on the derived category of coherent sheaves on cotangent bundles of Grassmannians