Obesity is characterized by adipose tissue (AT) macrophage (ATM) accumulation, which promotes AT inflammation and dysfunction. Toll-like receptor 4 (TLR4) deficiency attenuates AT inflammation in obesity but does not impede the accumulation of ATMs. The purpose of the current study was to determine whether TLR4 deficiency alters ATM polarization. TLR4^−/− and wild-type mice were fed a low-fat, high-monounsaturated fat (HF^MUFA), or a high-saturated fat (HF^SFA) diet for 16 weeks. Further, we used a bone marrow transplant model to determine the influence of hematopoietic cell TLR4 signaling. The metabolic and inflammatory responses to high-fat feeding and ATM phenotype were assessed. Global and hematopoietic cell TLR4 deficiency, irrespective of recipient genotype, produced a shift in ATM phenotype toward an alternatively activated state, which was accompanied by reduced AT inflammation. Despite the observed shift in ATM phenotype, neither global nor hematopoietic cell TLR4 deficiency influenced systemic insulin sensitivity after high-fat feeding. Results of the current study suggest that TLR4 directly influences ATM polarization but question the relevance of TLR4 signaling to systemic glucose homeostasis in obesity.