Rob's Tech Fun and Games

After some of the recent comments on various SharePoint posts, I thought it was worth going through the most important part of using SharePoint … planning!  SharePoint is a very powerful system, but it is really, really difficult to change the architecture once it’s already installed.  You NEED (and my apologies for shouting, but the emphasis is really deserved) to make sure that you install your directory services, security, SharePoint and prepare your site collections and templates correctly before you even think of using it.

This isn’t a step by step guide – SharePoint is simply too broad a platform for that sort of approach.  Instead, it lists some of the more common gotchas, or concepts to think about.  For more technical installation details, see my earlier posts:

Installing SharePoint 1 – Preinstallation

Installation SharePoint 2 – Installation

Installing SharePoint 3 – Configuration

I’m going to have lots of clients using my SharePoint system, but they can’t know about the others

Security within SharePoint is generally scoped at a Site Collection level.  If clients shouldn’t be able to see each other on the system, you need to plan for that before you deploy anything.  Every group of external users should be hosted on a separate site collection, or you will hit issues with the PeoplePicker.  If you intend to keep access control factors away from clients, thats not necessarily a show stopper, but as a general rule, use a site collection as a security scope.  You’ll also need to tell the PeoplePicker to be restrictive, using

stsadm -o setproperty –url http://<server> –pn peoplepicker-onlysearchwithinsitecollection –pv yes

If you find out that you have multiple clients using subsites on a site collection, its too late.  You’ll have to go back to the design phase, or start introducing technical kludges like replacing the PeoplePicker functionality yourself.

My network grew organically, and I currently have a fairly complex domain structure.

OK – stop right there!  If you have a complex domain structure, you need to think long and hard about which domains will access SharePoint, work out trusts and user rights, and manually configure SharePoint to be able to see all of the applicable Forests and Domains.  You’ll also need to work out any potential security headaches with firewalls, ports and domain authentication from your SharePoint installation.  Don’t install SharePoint, then think about this.  It’ll have all sorts of issues with the People Picker, the search services and user profiles, and with security and authentication too, especially if you hide SharePoint behind a security provider like ISA 2006!

I don’t want to add external users to my live Active Directory

Fair enough!  There are two possible solutions – either set up up a new Active Directory domain for external users in the DMZ, and then set up a one way trust (for internal users), or simply install another directory service.  ADAM is a light directory service, perfect for this sort of environment.  Either solution needs to be in place before you deploy SharePoint – again, preparation will stand you in great stead.

I’ve been running SharePoint for a while, and my security architecture is out of control.

This will happen, unless you have really done a good job in terms of planning ahead.  Wherever possible, try to ensure access rights are maintained as smoothly as possible.  If you set up roles based on directory service groups, then maintain those groups, your security takes care of itself, particularly for internal users, where your system administrators should maintain AD groups as a matter of course.  The more granular your application of security within SharePoint, the harder you’ll find maintenance.  Keep to groups, and maintain the groups wherever there isn’t a clear business need.

Final thoughts

Like any complex system, the key to Microsoft SharePoint is proper planning.  If you understand:

1. Your business problem (client restricted extranets are different to open forums, which differ from intranets!)
2. Your business environment (especially your technical deployment, such as DMZ requirements and domain structure, but a grasp of the basic business factors can be invaluable)
3. the architecture of Mcirosoft SharePoint

then you can deploy a system that will acheive your goal efficiently and securely.  If you install SharePoint without planning, because “SharePoint is the way to go”, with no understanding or foresight, then you’ll be problem solving until the project fails.

Tom commented on a post with the following problem, and I thought it merited a post.

“We have a MOSS 2007 FARM AND 3 DOMAINS all have a two way trust.  We have over 78 sites all of which stopped with no known reason from being able to find users that are in one of the domains we can find the users in the view profiles yet we can no longer find users using peoplepicker for any users from the one domain.

We have tried this command you have provided and they come back with commandline error
stsadm -o setproperty -pn peoplepicker-searchadforests -pv domain:full domain name,-userlogin domain\username password –url http://webapp url”

It sounds like an interesting problem, but its difficult to answer without more information.  Incidentally, the profiles comment, about being able to view user profiles, is rather a red herring.  This is handled by an import process specified elsewhere in the SSP, and has nothing to do with the People Picker displaying users.

Lets discuss the stsadm command first.   Without knowing the specific error message, I can’t say why the command is failing, but there are two probable outcomes.   stsadm is not generally included in the default path for a windows installation, so if the error message is:

’stsadm’ is not recognized as an internal or external command, operable program or batch file.

The problem is simply that you need to find the appropriate location first.  The location is generally:

C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN>

so simple change directory to that path, and run the STSADM again.  It sounds obvious, but I must admit, I had a bit of a nightmare trying to find the stsadm path when I first started looking at SharePoint!

The other possibility is that the command has been run, but that the parameters haven’t been entered correctly.  If so, you get the terribly helpful response of :

Command line error.

Followed by a complete syntax reference for the command, and it sounds much more likely that this is the cause of the problem, from the notes in the question.  Unfortunately, this is quite a lot harder to discuss, as unsuprisingly people aren’t going to give the specific command line with all of their configuration details and passwords to be put onto a website, and the generic, censored versions are probably going to be correct, at least as far as it goes.

The best help I can give here is to put together a full hypothetical example, rather than just repeating the command syntax yet again.

Essentially, you first need to set an internal SharePoint encryption key, then tell the server what domains to add to the list, and what valid username and password to use to connect to the domain in order to pull back the list.  Don’t use administrator, btw!!!

To set the initial encryption key, use:

stsadm.exe -o setapppassword -password <yourencryptionkey>

To set the actual domain link, use:

stsadm.exe -o setproperty -url http://domain1.example.com:80 -pn “peoplepicker-searchadforests” -pv “domain:domain1.example.com,domain1\LoginName, P@ssword; domain:domain2.example.com,domain2\LoginName, P@ssword; domain:domain3.example.com,domain3\LoginName, P@ssword“

Of course, with two way trusts a single user name and password could be used if you granted the appropriate rights. 

What normally goes wrong putting this together?  Normally it is either the encryption key hasn’t been set first, or that construction of the domain list has a syntax issue (or that the surrounding quotes have been missed off).

As a rule, though, if you get a command line error when running stsadm, you have got the format wrong.  If the format is right, it won’t necessarily solve the problem (if your username or password is wrong, for example, it still can’t access the other domain information), but you’ll see the changes applied.  A good way of checking is to run:

stsadm.exe -o getproperty -url http://domain1.example.com:80 -pn “peoplepicker-searchadforests”

And it will show the details you’ve set (with passwords asterisked out).

What makes the problem Tom is experiencing interesting is that apparently the People Picker has been working, and now isn’t.  This, to me, implies that something has changed with the installation, or the Active Directory configuration.

Key things to check would be:

Has the system user context still got access rights to the domain that appears to be no longer accessible?  A two way trust means that user rights can be assigned … by default they arent.

Have the active directory servers changed?  If so, you may need to force DNS updates, otherwise resolution against the AD may be looking for defunct servers.

Has the stsadm command been run succesfully in the past, and the usernames and passwords have since changed (or expired)?  This will obviouslly drop off the people pickers ability to query the domain.

Has a security patch been applied against the domain, or has permissions to the Active Directory been changed?  By default, older systems allowed anyone to do a basic LDAP query against the Active Directory, but this was locked down.  If this loophole was previously being used instead of setting correct access rights for the security context of the wep application, it’ll obviously start causing this issue.

I hope this helps – Tom, if you’ve solved the problem, please let me know the solution

OK, before I get started here, let me clarify.  I’m predominantly a SharePoint administrator, not a developer, so these reviews are based around their use for installation and administration, as opposed to their use for a web developer!  If I think something would be useful for different roles, I’ll try and point it out, but do bear the viewpoint in mind as you read further.

With the disclaimer out of the way, lets get started with O’Reilly’s “Essential SharePoint 2007″ (You can find this at Amazon.co.uk and Amazon.com).  I’ve got mixed feelings about this book.  I used it a lot when getting started with SharePoint, and its a great reference when I go back to do something that I’m not doing frequently.  However, it lacks depth, and seems a bit unfocussed, covering areas that developers, administrators and users all need to know.  Almost the entire team I work with, both developers and admins, have a copy of the book and use it on occasion, but its not a great place to start, and not the best place to go for a detailed technical reference either.  I’d say its absolutely ideal for someone tackling SharePoint in a smaller company, where you need to have an understanding across the board, rather than specialising in a particular area.  In a larger company, its a great book if you occasionally work with SharePoint and want a solid technical reminder.

Surprisingly, one of the most useful books for me for SharePoint was  “Microsoft SharePoint 2007 for Dummies” (You can find this at Amazon.co.uk and Amazon.com).  This may be because I’m a dummy, or just my method of approaching a new technology.  I read through a simple introduction to the technology, to get a rough idea of how it works, what the components are, how they fit together, and what the terminology is.  Because I start with a simple, clear and concise book , I can pick up that information really quickly, then I can use that information to bootstrap myself up to the complex technical specifics, and actually get the most of the in depth technical references.  The “Microsoft SharePoint 2007 for Dummies” is perfect for this, providing a really solid introduction to the concepts and terminology.  I think it’d also be useful for a developer trying to understand how SharePoint fits together and is likely to be used, and very useful for a technical manager who just needs broad brushstrokes while his team does the detail work.

I wasn’t keen on the “Microsoft Office SharePoint Server 2007 Administrator’s Companion” (You can find this at Amazon.co.uk and Amazon.com).    Others may feel differently about it – many of the technical people I work with love the style in these Microsoft Press books.  I find that its too focussed on how to achieve specific goals by clicking specific buttons, rather than focussing on the particularly settings you need, and why they are needed.  Its more a philosophical difference than anything – I’m not just focussed on results, I need to know why I’m doing something.  I’ve found a solid understanding of a technology leads to much better results … and a much better ability to troubleshoot issues … than simply knowing what you need to click to achieve a particular goal.  Unfortunately, that’s the path Microsoft in general have taken with their books and courses, and increasingly seems to be the approach taken by the technical people I meet.  If thats what you want, this book is perfect.  I didn’t get on with it.  If you check the Amazon reviews, however, you’ll find most people love it.

“SharePoint 2007 The Definitive Guide” (You can find this at Amazon.co.uk and Amazon.com) was pretty good as a guide.  Some aspects are very good – its coverage of SharePoint installations and upgrades from 2003 is excellent, for example.  Its coverage of network topologies and security is also excellent.  Where this falls down, and where the Internet shines, is the fact that I’ve found you need to know how a range of technologies work together to cover many of the odder demands of a SharePoint installation.  If you publish a SharePoint site, you’ll probably want to use ISA Server 2006 to secure it, not just rely on SharePoint.  Using a SQL server on a different domain via SQL authentication isn’t uncommon for DMZ deployments.  This book is great for vanilla SharePoint installs, and is definitive for basic admin tasks, such as deploying and configuring basic sites, roles and permissions.  You’ll need to look elsewhere for any installs beyond the vanilla.  I rarely go back to this book, but I’d thoroughly recommend reading through it at least once, and using it as a reference if you’re actively looking after SharePoint sites, as opposed to looking after the architecture.  If you need to get to grips at level of the stsadm command line, you won’t find it here.

As a follow up query to the previous notes on working with the PeoplePicker and ADAM, I’ve been asked about the behaviour of the PeoplePicker – specifically, that it appears to only return external users if their username is specifically searched for, not if part of their name is entered.  Is there a way to get exactly the same results for users from AD and ADAM?  Unfortunately, there are pretty severe limits to this, and unfortunately, I don’t believe you can actually get the same results, though I’d love to be proven wrong.

By default, when dealing with custom authentication sources, including ADAM, the PeoplePicker only returns exact matches.  If I search for the username, it will find it.  If its a user already on the site collection, it can use the local user details.  However, to find non exact matches in a custom repository, you need to edit the WebConfig file, by adding the following section:

<PeoplePickerWildcards>
  <clear />
  <add key=”ADAMMembership” value=”*” />
</PeoplePickerWildcards>

Unfortunately, this solution isn’t perfect.  It effectively adds the wildcard symbol * to every search in the people picker.  Lets illustrate this step by step, using a Pat Smith as an example name.

If I search for “Pat”, before updating the web.config file, they’ll only appear with the full name in the results if they’ve already been added to the site somewhere, or if they’re actually in the Active Directory repository.

If I search for “Pat” after updating the web.config file, all the Pats from the ADAM repository and from AD will appear.   Problem solved?  No.

If I search for Smith, as a surname, I’ll get all the Smiths from AD, but not from ADAM.   Thats because the search going to ADAM is actually like “Smith*”.  It’ll find everything starting with Smith, not all names containing Smith, and I haven’t come across any variations that will actually resolve this search issue.  However, you’ll probably find that this is a significant step forward in any event.

As always with SharePoint, make sure you edit all of the relevant web.config files.  You may find SharePoint is happier if you run

iisreset -noforce

after making the change, although in theory you shouldn’t need to - as a rule of thumb, major changes to web applications within SharePoint can have odd consequences otherwise.

In a stroke of rare genius, applying SharePoint SP2 to SharePoint 2007 has a strange effect – it resets the license type to become a 180 day trial version!

Its not a major issue – reapplying your license key in the “Convert License Type” section in Central Administration will reset it, and you won’t have lost any data in the meantime, but its something you need to be aware of…. otherwise you might hit more than a few issues in about 6 months!

One of the comments on my PeoplePicker post asked some questions about the way PeoplePicker works with ECTS – the External Collaboration Toolkit for SharePoint.  To be honest, I haven’t worked with the ECTS myself, but I understand the theory.

The ECTS uses Microsoft’s ADAM – Active Directory Application Mode – to act a a user repository, which gives you a SharePoint structure that should resemble the following:

External Collaboration Toolkit for SharePoint Architecture

Getting the PeoplePicker to work correctly with this is difficult, but not impossible.  The question asked was what could cause the PeoplePicker to fail to return AD users when logged in as an external user?   Of course, as is often the case with comments on blogs, there really isn’t anywhere near enough information about the setup to answer accurately.

My first guess would be that the PeoplePicker has actually been deliberately configured that way.  It is a potentially huge security risk to allow external users to see all the usernames of your company, which is why many people using ADAM authentication run the following command:

stsadm -o setproperty -url https://<url> -pn peoplepicker-nowindowsaccountsfornonwindowsauthenticationmode -pv yes

This command deliberately stops PeoplePicker returning internal AD users when logged in via forms based authentication outside of windows, including using ADAM.  You can tell if this is turned on by running:

stsadm -o getproperty -url https://<url> -pn peoplepicker-nowindowsaccountsfornonwindowsauthenticationmode

If this is the problem, you can turn the security feature off by running:

stsadm -o setproperty -url https://<url> -pn peoplepicker-nowindowsaccountsfornonwindowsauthenticationmode -pv no

And your PeoplePicker should leap back into life for all users.

However, there are still some quirks with using the PeoplePicker with ADAM, even if AD users are allowed to respond.  Take a look at Matt Morse’s excellent blog on how the PeoplePicker returns different results from ADAM than you might expect if you are used to working with normal windows authentication.

EDIT:-

As always, after applying stsadm commands to your SharePoint installation, don’t forget to either reset IIS or the relevant application pool.

After installing SharePoint, and importing all the User Profiles, you’ll find if you are using the People Picker, you’ll only see users from the trusted domain that have successfully logged on the SharePoint 2007 server, rather than all of them.  This is rather bizarre, as you’ve imported all the profiles, and can see them!

You actually need to configure the PeoplePicker to do a lookup to the Domain Controllers, as its here the People looks, NOT the SharePoint user profile store.  It seems unusual, but true!  It’s quite a complex task.

Before you do anything else, work out a list of all of the domains the PeoplePicker needs to look at, INCLUDING the domain SharePoint is installed on.  If SharePoint is on Domain1 and you want to see all the Domain1 users and all the users from the trusted domain Domain2, you’re going to need to list them both, something most of the guides online don’t make clear.  You’ll also need the fully qualified domain names – doing use the older NetBIOS name.  If your domain is exampledomain.local, don’t just use “exampledomain” – we’ll need the full thing.

Next, make sure you have a valid Active Directory user account on each of the domains you want to look at.  You don’t need to worry about the domain the SharePoint server is on – the accounts SharePoint should be running under will already have access.

Now, we first need to set up an encryption key, so SharePoint can securely store the usernames and passwords for the other domains.  Use the following command on every server in the farm – if you don’t, the other SharePoint servers won’t be able to decrypt the stored user names and passwords:

stsadm –o setapppassword –password MyPassword

Replace MyPassword with your chosen encryption key, of course! 

Next, we need to tell each Web Front End server, which domains to use.  I always list the current domain SharePoint is a member of first, for ease of reference.  Normally, I’d expect at least two entries – the current domain and the trusted domain (or domains) – if there isn’t a trusted domain, why are you doing this???.  We’ll need to separate the entries in the domain list with semi-colons.

stsadm -o setproperty -pn peoplepicker-searchadforests -pv domain:domain1.com;domain:domain2.com,domain2\user,password –url https://sharepoint.domain1.com

Here, the url should be replaced by that of your web application – don’t forget to use https if you’ve set the application up to use SSL.  Domain names should obviously be replaced with your own, and you should use the usernames and passwords from each domain that you either created or ensured were available earlier.  A more realistic looking example might be:

stsadm -o setproperty -pn peoplepicker-searchadforests -pv domain:technet.microsoft.com;domain:kb.microsoft.com,kb\AD_Lookup,LookUp2009 –url https://sharepoint.technet.microsoft.com

Please note this is an entirely hypothetical example, so don’t think of trying the links or usernames! 

Fianlly, though this generally isn’t mentioned in most of the other guides, you need to reset IIS before SharePoint will pick up the changes.  As always, I prefer the noforce option, just in case.

issreset -noforce

You should now see all of the available people from the domains you’ve selected in the people picker!

Publishing SharePoint via ISA Server is fairly straightforward, even if you use SSL to the reverse proxy, and connect on port 80 to the SharePoint server in order to move the encryption load from your SharePoint box.

However, for some reason, this appears to break Telerik’s RadEditor.  In order to use RadEditor, you need to be able to connect directly to the server in the same format of the initial request, which means in the case of a secured SharePoint site, SSL.   Simply change the bridging rule to redirect requests to the SSL port, and disable redirection to the HTTP port.  This will place more load on your SharePoint servers – every request is encrypted via SSL between the SharePoint server and the proxy – weigh this load up against the editing benefits of RadEditor.

One of the common problems I’ve seen with Microsoft SharePoint is the loss of search functionality, and I’ve found a lot of different theories and possible solutions.  I’ve tried to combine the various ideas into a step by step troubleshooting strategy.  Pleae note that this only applies if you’ve had a working earch service in the past – if not, you need to enable the search service before you do anything else!

Symptoms

Individual sites and site collections respond to any search query with no results.

The event log on the SharePoint Server shows Event  ID 2436 – The start address <site url> cannot be crawled.

The crawl log within SharePoint says that the site cannot be crawled and has been deleted from the gatherer.

Possible Resolutions

First, disable loopback checking on IIS, if you haven’t already.  This is one of the most common causes of the problem, and if it isn’t done, means that you WILL experience the problem sooner or later anyway.  To disable loopback checking, you need to make a registry change and restart your server, so schedule it for a quiet time.

Follow these steps:

1. Click Start, click Run, type regedit, and then click OK.
2. In Registry Editor, locate and then click the following registry key:
   HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. Right-click Lsa, point to New, and then click DWORD Value.
4. Type DisableLoopbackCheck, and then press ENTER.
5. Right-click DisableLoopbackCheck, and then click Modify.
6. In the Value data box, type 1, and then click OK.
7. Quit Registry Editor, and then restart your computer.

Second, check the permissions on the search service.  It sounds silly, but it is easy to sometimes use an account that doesn’t have access rights to actually crawl the sites!  Don’t get too involved with this, however – if it looks right, it probably is.  Its easy to waste days checking odd possibilities of access rights, but if this is the cause, its normally pretty obvious!

Third, ensure that there is a site at the top level of the web application.  This sounds ludicrous, but I’ve seen systems spring into life after a blank site is deployed at the root of the website.  Its easy to check, and easy to fix.  Many people won’t have seen this, as it’s pretty common practise to deploy self service site creation in the root url.  I’ve particularly seen this on systems after an SP2 MOSS and WSS install.  If a 404 error is received on the root url, all the other sites won’t be crawled.