Math.pku.edu.cn

A note on permutation groups with a regular subgroup Dedicated to Cheryl Praeger on the occasion of her 60th birthday In this note we first prove that for a positive integer n > 1 with n = p or p2 where p is a prime, there exists a transitive group of degree n without regular subgroups.
Then we look at 2-closed transitive groups without regular subgroups, and pose twoquestions and a problem for further study.
We first define four subsets of positive integers: N R = {n ∈ N there is a transitive group of degree n without a regular subgroup} N2R = {n ∈ N there is a 2-closed transitive group of degree n without a regular subgroup} N D = {n ∈ N there is a vertex-transitive digraph of order n which is non-Cayley} = {n ∈ N there is a vertex-transitive graph of order n which is non-Cayley} In the literature there is much work studying the set N C; see [5, 6, 7, 8, 9] for example.
∈ N C by [7, Theorem 3], but 12 ∈ N R since M11, acting on 12 points, has no regular subgroup by [3]. And it is easy to see that 6 is the smallest number in N R \ N C since A6has no regular subgroups. In the first part of this note we shall determine the set N R.
It is well-known that any prime number p does not belong to any one of the four sets above. Moreover, Maruˇsiˇc [5] proved that p2 / ∈ N C. In fact, we have p2 / Proposition 1 Any transitive group G of degree p2 on Ω has a regular subgroup. Hencep2 / Proof Take a minimal transitive subgroup P of G. Then P is a p-group and every maximalsubgroup M of P is intransitive. For any α ∈ Ω, we have |Pα| = |P |/p2 and |Mα| > |M|/p2,so Mα = Pα. It follows that Pα ≤ M and hence Pα ≤ Φ(P ). If |P : Φ(P )| = p, then P iscyclic and is regular. If |P : Φ(P )| = p2, then Pα = Φ(P ). Since Φ(P ) is normal in P andPα is core-free, we have Pα = 1 and hence P ∼ The following example shows that p3 ∈ N R. However, it was proved that p3 / [5, 6]. Therefore p3 ∈ N R \ N C.
Example 2 (1) Let p be an odd prime and let G be the group of order p4 presented by G = a, b ap2 = bp = cp = 1, [a, b] = c, [c, a] = ap, [c, b] = 1 . Let H = c . Consider the transitive permutation representation ϕ of G acting on the cosetspace [G : H]. Then ϕ(G) is a transitive group of degree p3, and ϕ(G) has no regularsubgroups. G = a, b, c, d a2 = b2 = c2 = d4 = 1, [a, b] = [b, c] = [c, a] = 1, ad = ab, bd = bc, cd = c . Z4 has order 25. Let H = b, d2 and ϕ be the transitive permutation representation of G acting on the coset space [G : H]. Then ϕ(G) is a transitive group ofdegree 23 and has no regular subgroup. Proof (1) Since [c, a] = ap, c G. Since Ker ϕ = coreG(H) = 1, the action is faithful.
= G. Suppose that ϕ(G) has a regular subgroup, say ϕ(R). Then R is maximal in G, and RH = G by the Frattini argument. However, H ≤ G ≤ Φ(G) ≤ R, a contradiction.
(2) Similar to (1), prove that H is core-free and contained in Φ(G). Details omitted. ✷ Now we are ready to determine the set N R. We first need the following proposition.
Proposition 3 Let p < q be two primes. Then pq ∈ N R. Proof Let W = Zp Zq = a b , viewed as an imprimitive group of degree pq. Since the action of b on the base group Zqp is nontrivial, we may take a b -invariant subgroup H of the base group such that the action of b on H is also nontrivial and H is smallest subjectto this property. Then b is irreducible on H. Let G = H b . Since p < q, |H| = pk > p.
H. Consider the transitive permutation representation ϕ of G acting on the coset space [G : M ]. Since H is a minimal normal subgroup of G, coreG(M) = 1 and ϕ is faithful.
Since b is a Sylow q-subgroup and maximal in G by the irreducibility of b on H, G has nosubgroup of order pq. Hence ϕ(G) has no regular subgroups. It follows that pq ∈ N R. ✷ Theorem 4 Let n be a positive integer greater than 1. Then n ∈ N R unless n = p or p2for a prime p. Proof This theorem follows from Proposition 1, Example 2, Proposition 3 and the factthat if m ∈ N R then km ∈ N R for any positive integer k.
In the second part of this note we look at the set N2R. The next proposition shows that ∈ N2R, while Maruˇsiˇc [5] proved that p3 / Proposition 5 Any 2-closed transitive group G of degree p3 on Ω has a regular subgroup. To prove the above proposition, we need the concept of 2-closures of permutation groups Let G be a permutation group acting on Ω. Suppose that ∆0, ∆1, . . . , ∆r−1, are orbits of G acting on Ω × Ω. The 2-closure G(2) of G is defined by G(2) = {x ∈ Sym(Ω)|∆xi = ∆i, i = 0, 1, . . . , r − 1}. Obviously, G(2) ≥ G; if G(2) = G, we say that G is 2-closed. The following lemma is quotedfrom [10, Exercise 5.28].
Lemma A: Suppose that G is a 2-closed group and p a prime. Then the Sylow p-subgroupP of G is also 2-closed. Theorem B: (Wielandt’s dissection theorem) Let G be a permutation group acting on Ω,and H a subgroup of G. Suppose that Ω = ∆ ∪ Γ, ∆ ∩ Γ = ∅, ∆ = ∅, Γ = ∅ and ∆H =∆, ΓH = Γ. If for any δ ∈ ∆, H and Hδ have the same orbits on Γ, then H∆ × HΓ ≤ G(2). This theorem follows from [10, Theorem 6.5] and the following obvious fact: if H ≤ G, Let P ∈ Syl(G). Then P is also transitive on Ω. Take an element z ∈ Z(P ) with o(z) = p. Let B = {B1, . . . , Bp2} be the set of orbits of z . Then Bis a complete block system of P . Assume that K = PB is the kernel of P acting on B. SinceKBi = Zp, K is elementary abelian. Set P = P/K. P is a transitive group on B.
Take 1 = x ∈ K such that the support supp(x) of x has the minimum size. We claim that supp(x) is a block of P . Since K is elementary abelian, x is of order p. If supp(x)were not a block of P , then we could find an h ∈ P such that supp(x)h = supp(x) andD = supp(x) ∩ supp(x)h = ∅. Since every Bi is a block of size p and p a prime, supp(x),supp(x)h = supp(xh) and D are unions of several entire blocks of P in B. Set J = x, xh .
Then the nontrivial orbits of J are precisely the blocks contained in supp(x) ∪ supp(xh).
It is not difficult to check that for any g ∈ Ω − D, J and Jg have the same orbits in D.
So, by Theorem B, JD × 1Ω−D ≤ P (2). (In what follows, for brevity, we use JD to denoteJD × 1Ω−D.) Since P is 2-closed, P = P (2). Then JD ≤ P and hence JD ≤ K. Thus thereexists an element y ∈ JD ≤ K of order p such that |supp(y)| < |supp(x)|, which contradictsthe choice of x.
Now we distinguish three cases: (1) |supp(x)| = p; (2) |supp(x)| = p2 and (3) |supp(x)| = If |supp(x)| = p, then supp(x) = Bi for some i. Hence P = Zp P . Since P is a transitive group of degree p2, it has a regular subgroup H isomorphic to Z2p or Zp2. Thus P has a subgroup isomorphic to Zp H, which has an abelian regular subgroup.
If |supp(x)| = p3, then K = z is semiregular. Assume that H/K is a regular subgroup of P = P/K. Then H is a regular subgroup of P .
Finally we assume that |supp(x)| = p2. Then supp(x) is a block of P of length p2 which is a union of p Bis. Assume that C = {C1, . . . , Cp} is a block system of P with supp(x) = C1.
Then K = KC1 × · · · × KCp ∼ If P = P/K has an element aK of order p2, then a, z is a regular subgroup of P .
So we may assume that expP = p. Take a regular subgroup H/K = uK, vK of P suchthat v ∈ PC − K. (Since PC has index p in P , this is possible.) We have HC = v, K andu / ∈ HC. Without loss of generality we assume that Cui = Ci+1, for all i (mod p). Define a w = vC1(vu)C2(vu2)C3 · · · (vup−1)Cp. Since [v, u] = k ∈ K, vu = vk. So w = vC1(vk)C2(vkku)C3 · · · (vkku · · · kup−2)Cp = (vC1vC2vC3 · · · vCp)(1C1kC2(kku)C3 · · · (kku · · · kup−2)Cp)= v¯ k = 1C1kC2(kku)C3 · · · (kku · · · kup−2)Cp ∈ K, as K = KC1 × · · · × KCp. So w ∈ HC.
Since up ∈ K, vup = v. So u centralizes w, and then R = u, w is abelian and RK/K istransitive on B. If |R| > p2, then R is regular; if |R| = p2, then R × z is regular. Thiscompletes the proof of this proposition.
It is known that not all 2-closed transitive groups are the full automorphism groups of a (di)graph. For example, the regular representation of a finite group that has no GRR orDRR is such an example since regular groups are obviously 2-closed. (For GRRs and DRRsof finite groups, see [1, 2, 4].) Now we would like to pose the following questions.
Question 1 Is N2R = N C? To study Question 1, we should first find nonregular 2-closed groups which are not the full automorphism groups of (di)graphs. We do not know such examples.
To end this note, we propose a problem, we first define one more subset of positive PN R = {n ∈ N there is a primitive group of degree n without a regular subgroup} Different from the set N R, we know that pn / ∈ PN R for any prime p and any positive integer n; see [11, Theorem]. Hence, determining the set PN R should be much harder thanN R.
The author is grateful to Dr. Haipeng Qu and Dr. Jing Xu for their valuable discussions This work was supported by NSFC (proj. no. 10671114).
[1] L´aszl´o Babai, On the abstract group of automorphisms, Combinatorics (Ed. H. N. V.
Temperley), London Math. Soc. Lecture Note Series 52, 1981; 1–40.
[2] L´aszl´o Babai, Finite digraphs with regular automorphism groups, Periodica Math. [3] Michael Giudici, Factorisations of sporadic simple groups, J. Algebra, 304 (2006), 311– [4] Chris D. Godsil, GRR’s for non-solvable groups, Colloq. Math. Soc. Jannos Bolyai, [5] Dragan Maruˇsiˇc, Vertex-transitive graphs and digraphs of order pk, Ann. of Discrete [6] Brendan D. McKay, Cheryl E. Praeger, Vertex-transitive graphs which are not Cayley graphs, I, J. Austral. Math. Soc.(A) 56(1994), 53–63.
[7] Brendan D. McKay, Cheryl E. Praeger, Vertex-transitive graphs which are not Cayley graphs, II, J. Graph Theory 22(1996), 321–334.
[8] A. A. Miller, Cheryl E. Praeger, Non-Cayley, vertex transitive graphs of order twice the product of two distinct odd primes, J. Alg. Combin. 3(1994), 77–111.
[9] M. A. Iranmanesh, Cheryl E. Praeger, On non-Cayley vertex-transitive graphs of order a product of three primes, J. Combin. Theory Ser. B 81(2001), 1–19.
[10] H. Wielandt, Permutation Groups Through Invariant Relations And Invariant Func- tions, Ohio State University, Columbus, 1969.
[11] Ming-Yao Xu, Vertex-primitive digraphs of prime-power order are hamiltonian, Discrete

Source: http://www.math.pku.edu.cn:8000/var/preprint/7209.pdf