Cook the bacon in a heavy pot over medium-high heat, stirring occasionally, until crisp, 12 to 15 minutes. Using a slotted spoon, transfer the bacon to a paper towel–lined plate to cool.
While the bacon cooks, thinly slice the scallions, reserving the green and white parts separately.
Once the bacon is removed from the pot, add the butter, celery and scallion whites and cook over medium heat, stirring occasionally, until softened, about 10 minutes.
Add the potatoes, thyme, chicken broth, 1 teaspoon salt and ½ teaspoon pepper and bring to a simmer. Simmer until the potatoes are very tender, 30 to 40 minutes.
Remove and discard the thyme sprig. Purée the soup in a blender until very smooth (use caution when blending hot liquids; alternatively, use a hand blender). Return the soup to the pot and whisk in the milk or cream and season to taste with salt and pepper, thinning as needed. Reheat gently over medium-low heat.
Divide the soup among warm bowls and top with the crisp bacon and reserved scallion greens.