In this case, the form of $\psi \,$ is not obvious and has to be
derived from the traction-free BCs

$(\psi _{,1}-x_{2}){\hat {n}}_{1}+(\psi _{,2}+x_{1}){\hat {n}}_{2}=0~~~~\forall (x_{1},x_{2})\in \partial {\text{S}}$ Suppose that $2a\,$ and $2b\,$ are the two sides of the rectangle, and $a>b\,$ .
Also $a\,$ is the side parallel to $x_{1}\,$ and $b\,$ is the side parallel to $x_{2}\,$ .
Then, the traction-free BCs are

$\psi _{,1}=x_{2}~~{\text{on}}~~x_{1}=\pm a~~,{\text{and}}~~\psi _{,2}=-x_{1}~~{\text{on}}~~x_{2}=\pm b\,$ A suitable $\psi \,$ must satisfy these BCs and $\nabla ^{2}{\psi }=0\,$ .

We can simplify the problem by a change of variable

${\bar {\psi }}=x_{1}~x_{2}-\psi$ Then the equilibrium condition becomes

$\nabla ^{2}{\bar {\psi }}=0$ The traction-free BCs become

${\bar {\psi }}_{,1}=0~~{\text{on}}~~x_{1}=\pm a~~,{\text{and}}~~{\bar {\psi }}_{,2}=2x_{1}~~{\text{on}}~~x_{2}=\pm b$ Let us assume that

${\bar {\psi }}(x_{1},x_{2})=f(x_{1})g(x_{2})$ Then,

$\nabla ^{2}{\bar {\psi }}={\bar {\psi }}_{,11}+{\bar {\psi }}_{,22}=f^{''}(x_{1})g(x_{2})+g^{''}(x_{2})f(x_{1})=0$ or,

${\frac {f^{''}(x_{1})}{f(x_{1})}}=-{\frac {g^{''}(x_{2})}{g(x_{2})}}=\eta$ Case 1: η > 0 or η = 0
edit
In both these cases, we get trivial values of $C_{1}=C_{2}=0\,$ .

Case 2: η < 0
edit
Let

$\eta =-k^{2}~~~;~~k>0$ Then,

${\begin{aligned}f^{''}(x_{1})+k^{2}f(x_{1})=0~~\Rightarrow &~~f(x_{1})=C_{1}\cos(kx_{1})+C_{2}\sin(kx_{1})\\g^{''}(x_{2})-k^{2}g(x_{2})=0~~\Rightarrow &~~g(x_{2})=C_{3}\cosh(kx_{2})+C_{4}\sinh(kx_{2})\end{aligned}}$ Therefore,

${\bar {\psi }}(x_{1},x_{2})=\left[C_{1}\cos(kx_{1})+C_{2}\sin(kx_{1})\right]\left[C_{3}\cosh(kx_{2})+C_{4}\sinh(kx_{2})\right]$ Apply the BCs at $x_{2}=\pm b\,$ ~~ (${\bar {\psi }}_{,2}=2x_{1}$ ), to get

${\begin{aligned}\left[C_{1}\cos(kx_{1})+C_{2}\sin(kx_{1})\right]\left[C_{3}\sinh(kb)+C_{4}\cosh(kb)\right]&=2x_{1}\\\left[C_{1}\cos(kx_{1})+C_{2}\sin(kx_{1})\right]\left[-C_{3}\sinh(kb)+C_{4}\cosh(kb)\right]&=2x_{1}\end{aligned}}$ or,

$F(x_{1})G^{'}(b)=2x_{1}~~;~~~F(x_{1})G^{'}(-b)=2x_{1}\,$ The RHS of both equations are odd. Therefore, $F(x_{1})$ is odd. Since,
$\cos(kx_{1})\,$ is an even function, we must have $C_{1}=0\,$ .

Also,

$F(x_{1})\left[G^{'}(b)-G^{'}(-b)\right]=0$ Hence, $G'(b)\,$ is even. Since $\sinh(kb)\,$ is an odd function, we must
have $C_{3}=0\,$ .

Therefore,

${\bar {\psi }}(x_{1},x_{2})=C_{2}C_{4}\sin(kx_{1})\sinh(kx_{2})=A\sin(kx_{1})\sinh(kx_{2})$ Apply BCs at $x_{1}=\pm a\,$ (${\bar {\psi }}_{,1}=0$ ), to get

$Ak\cos(ka)\sinh(kx_{2})=0\,$ The only nontrivial solution is obtained when $\cos(ka)=0$ , which means that

$k_{n}={\frac {(2n+1)\pi }{2a}}~~,~~~n=0,1,2,...$ The BCs at $x_{1}=\pm a\,$ are satisfied by every terms of the series

${\bar {\psi }}(x_{1},x_{2})=\sum _{n=0}^{\infty }A_{n}\sin(k_{n}x_{1})\sinh(k_{n}x_{2})$ Applying the BCs at $x_{1}=\pm b\,$ again, we get

$\sum _{n=0}^{\infty }A_{n}k_{n}\sin(k_{n}x_{1})\cosh(k_{n}b)=2x_{1}~~\Rightarrow ~~~\sum _{n=0}^{\infty }B_{n}\sin(k_{n}x_{1})=2x_{1}$ Using the orthogonality of terms of the sine series,

$\int _{-a}^{a}\sin(k_{n}x_{1})\sin(k_{m}x_{1})dx_{1}={\begin{cases}0&{\rm {if}}~m\neq n\\a&{\rm {if}}~m=n\end{cases}}$ we have

$\int _{-a}^{a}\left[\sum _{n=0}^{\infty }B_{n}\sin(k_{n}x_{1})\right]\sin(k_{m}x_{1})dx_{1}=\int _{-a}^{a}\left[2x_{1}\right]\sin(k_{m}x_{1})dx_{1}$ or,

$B_{m}a={\frac {4}{ak_{m}^{2}}}\sin(k_{m}a)$ Now,

$\sin(k_{m}a)=\sin \left({\frac {(2m+1)\pi }{2}}\right)=(-1)^{m}$ Therefore,

$A_{m}={\frac {B_{m}}{k_{m}\cosh(k_{m}b)}}={\frac {(-1)^{m}32a^{2}}{(2m+1)^{3}\pi ^{3}\cosh(k_{m}b)}}$ The warping function is

$\psi =x_{1}x_{2}-{\frac {32a^{2}}{\pi ^{3}}}\sum _{n=0}^{\infty }{\frac {(-1)^{n}\sin(k_{n}x_{1})\sinh(k_{n}x_{2})}{(2n+1)^{3}\cosh(k_{n}b)}}$ The torsion constant and the stresses can be calculated from $\psi$ .