The Fermi Gamma-ray Burst Monitor (GBM) triggers on-board in response to $sim$ 40 short gamma-ray bursts (SGRBs) per year; however, their large localization regions have made the search for optical counterparts a challenging endeavour. We have developed and executed an extensive program with the wide field of view of the Zwicky Transient Facility (ZTF) camera, mounted on the Palomar 48 inch Oschin telescope (P48), to perform target-of-opportunity (ToO) observations on 10 Fermi-GBM SGRBs during 2018 and 2020-2021. Bridging the large sky areas with small field of view optical telescopes in order to track the evolution of potential candidates, we look for the elusive SGRB afterglows and kilonovae (KNe) associated with these high-energy events. No counterpart has yet been found, even though more than 10 ground based telescopes, part of the Global Relay of Observatories Watching Transients Happen (GROWTH) network, have taken part in these efforts. The candidate selection procedure and the follow-up strategy have shown that ZTF is an efficient instrument for searching for poorly localized SGRBs, retrieving a reasonable number of candidates to follow-up and showing promising capabilities as the community approaches the multi-messenger era. Based on the median limiting magnitude of ZTF, our searches would have been able to retrieve a GW170817-like event up to $sim$ 200 Mpc and SGRB afterglows to z = 0.16 or 0.4, depending on the assumed underlying energy model. Future ToOs will expand the horizon to z = 0.2 and 0.7 respectively.