We describe multiscale geometrical changes via structured deformations ( g , G ) and the non-local energetic response at a point x via a function Ψ of the weighted averages of the jumps [ u n ] ( y ) of microlevel deformations u n at points y within a distance r of x. The deformations u n are chosen so that lim n → ∞ u n = g and lim n → ∞ ∇ u n = G . We provide conditions on Ψ under which the upscaling “ n → ∞ ” results in a macroscale energy that depends through Ψ on (1) the jumps [ g ] of g and the “disarrangement field” ∇ g − G , (2) the “horizon”r, and (3) the weighting function α r for microlevel averaging of [ u n ] ( y ) . We also study the upscaling “ n → ∞ ” followed by spatial localization “ r → 0 ” and show that this succession of processes results in a purely local macroscale energy I ( g , G ) that depends through Ψ upon the jumps [ g ] of g and the “disarrangement field” ∇ g − G alone. In special settings, such macroscale energies I ( g , G ) have been shown to support the phenomena of yielding and hysteresis, and our results provide a broader setting for studying such yielding and hysteresis. As an illustration, we apply our results in the context of the plasticity of single crystals.