Probabilistic graphical models are a marriage of graph theory and probability theory that are being widely used as statistical models in a variety of problem domains, including information retrieval, networking, natural language processing, bioinformatics, coding and signal processing. A recurring issue in using these models is that of specifying the structure of the models; an issue that often surfaces under the rubrique of "smoothing," "nuisance parameters" or "robustness". In this talk I present two kinds of approaches to dealing with this issue. The first is an "assumption-laden" empirical Bayesian approach, which I illustrate with a class of latent variable models for finding structure in document collections, and for annotating images from captions. The second is an "assumption-free" semiparametric approach, which I illustrate with a new class of models for finding independent and conditionally-independent structure in signals.